DRAFT [QA-TICKET] Slack test results notification fails to report failed E2E test #4833
Conversation
aweng98
changed the title
.circleci/wait-for-job-finish.sh
Outdated
| runs=$(echo "$job_detail" | jq -r '.parallel_runs[]') | ||
| running_vm=$(echo "$runs" | jq -r 'select(.status=="running")') | ||
| count=$(echo "$running_vm" | grep -c -e "running" || test $? = 1;) |
There was a problem hiding this comment.
Break down into smaller jq statements instead one very long statement. Easier to read and debugging.
Removed select(.index != '"$CIRCLE_NODE_INDEX"')' condition. Is this going to be a problem for the while loop?
There was a problem hiding this comment.
Removed
select(.index != '"$CIRCLE_NODE_INDEX"')'condition. Is this going to be a problem for the while loop?
Yes, I think that's going to be a problem. This script can't wait for the runner that it is running on to finish because then the script will be waiting on itself.
There was a problem hiding this comment.
That makes sense! It has been too long since last time i looked at this file.
.circleci/wait-for-job-finish.sh
Outdated
| if [ "$count" -eq 0 ]; then | ||
| echo "Parallel runs have finished. Exiting wait-for-job-finish script." | ||
| echo "$runs" | ||
| exit 0 |
There was a problem hiding this comment.
Additional logs for debugging or manual inspection.
aweng98
changed the title
|
hypothesis, need testing.
Jira Ticket: https://broadworkbench.atlassian.net/browse/[Ticket #]
Summary of changes:
What
terra-ui integration tests Slack notification were not reporting failed tests in the notifications sometimes. Seeing in this job and job.
Why
I think it's a timing issue, occurs when the data update time in CircleCI varies. While CircleCI is in the middle of updating job data from remaining
CIRCLE_NODE_INDEX: 0node, the execution ofnotify-circleci-test-results.jshas prematurely gathered incomplete test results and finished slack notification.either
test-summary-0.jsondoes't show failed test because it was the "final" or this file could be missing and not read at all.Testing strategy