fix(analysis): prevent promotion on inconclusive background run. Fixes: #3850

[ypopivniak-cg]

The current patch is applied for background analysis runs. It fixes unexpected promotion of a Rollout on the canary pause step and background analysis run with Inconclusive results. Instead of promoting on Inconclusive result reconciler must keep Pause phase/status of the Rollout for manual intervention (promote/abort) similarly to step analysis runs.
In addition, the PR adds an end-to-end test for this specific edge case.

Fixes: #3850

Current behavior:

Name:            rollout-background-analysis-inconclusive
Namespace:       user-ypopivniak
Status:          ✔ Healthy
Strategy:        Canary
  Step:          2/2
  SetWeight:     100
  ActualWeight:  100
Images:          nginx:1.19-alpine (stable)
Replicas:
  Desired:       2
  Current:       2
  Updated:       2
  Ready:         2
  Available:     2

NAME                                                                  KIND         STATUS          AGE  INFO
⟳ rollout-background-analysis-inconclusive                            Rollout      ✔ Healthy       37s
├──# revision:2
│  ├──⧉ rollout-background-analysis-inconclusive-867588b79c           ReplicaSet   ✔ Healthy       36s  stable
│  │  ├──□ rollout-background-analysis-inconclusive-867588b79c-mrcdl  Pod          ✔ Running       36s  ready:1/1
│  │  └──□ rollout-background-analysis-inconclusive-867588b79c-ff52h  Pod          ✔ Running       5s   ready:1/1
│  ├──α rollout-background-analysis-inconclusive-867588b79c-2         AnalysisRun  ? Inconclusive  35s  ? 4
│  └──α rollout-background-analysis-inconclusive-867588b79c-2.1       AnalysisRun  ✔ Successful    14s  ? 2
└──# revision:1
   └──⧉ rollout-background-analysis-inconclusive-6854cdf769           ReplicaSet   • ScaledDown    37s

Expected behavior:

Name:            rollout-background-analysis-inconclusive
Namespace:       user-ypopivniak
Status:          ॥ Paused
Message:         InconclusiveAnalysisRun
Strategy:        Canary
  Step:          2/2
  SetWeight:     100
  ActualWeight:  100
Images:          nginx:1.19-alpine (canary, stable)
Replicas:
  Desired:       2
  Current:       3
  Updated:       1
  Ready:         3
  Available:     3

NAME                                                                  KIND         STATUS          AGE  INFO
⟳ rollout-background-analysis-inconclusive                            Rollout      ॥ Paused        35s
├──# revision:2
│  ├──⧉ rollout-background-analysis-inconclusive-587f48cf55           ReplicaSet   ✔ Healthy       34s  canary
│  │  └──□ rollout-background-analysis-inconclusive-587f48cf55-94vb5  Pod          ✔ Running       34s  ready:1/1
│  └──α rollout-background-analysis-inconclusive-587f48cf55-2         AnalysisRun  ? Inconclusive  33s  ? 4
└──# revision:1
   └──⧉ rollout-background-analysis-inconclusive-6854cdf769           ReplicaSet   ✔ Healthy       35s  stable
      ├──□ rollout-background-analysis-inconclusive-6854cdf769-7l9kr  Pod          ✔ Running       35s  ready:1/1
      └──□ rollout-background-analysis-inconclusive-6854cdf769-8lkfp  Pod          ✔ Running       35s  ready:1/1

To reproduce the described issue without a patch and with the new end-to-end test, it can be run separately on the commit:

git checkout ec03c7e094addebf1e850b3fccbf12edeafb9235 # Updated due to commits re-signing
E2E_TEST_OPTIONS="-run 'TestAnalysisSuite' -testify.m 'TestCanaryInconclusiveBackgroundAnalysis'" make test-e2e

Checklist:

• Either (a) I've created an enhancement proposal and discussed it with the community, (b) this is a bug fix, or (c) this is a chore.
• The title of the PR is (a) conventional with a list of types and scopes found here, (b) states what changed, and (c) suffixes the related issues number. E.g. "fix(controller): Updates such and such. Fixes #1234".
• I've signed my commits with DCO
• I have written unit and/or e2e tests for my change. PRs without these are unlikely to be merged.
• My builds are green. Try syncing with master if they are not.
• My organization is added to USERS.md.

[github-actions]

Published E2E Test Results

  4 files    4 suites   3h 23m 11s ⏱️
114 tests 102 ✅  7 💤 5 ❌
462 runs  429 ✅ 28 💤 5 ❌

For more details on these failures, see this check.

Results for commit b833028.

♻️ This comment has been updated with latest results.

[github-actions]

Published Unit Test Results

2 279 tests   2 279 ✅  2m 59s ⏱️
  128 suites      0 💤
    1 files        0 ❌

Results for commit b833028.

♻️ This comment has been updated with latest results.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 82.70%. Comparing base (1264f8d) to head (b833028).
Report is 7 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3873      +/-   ##
==========================================
+ Coverage   82.69%   82.70%   +0.01%     
==========================================
  Files         163      163              
  Lines       22886    22889       +3     
==========================================
+ Hits        18925    18931       +6     
+ Misses       3087     3084       -3     
  Partials      874      874              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ypopivniak-cg]

Seems like one of the non-required tests failed because of the 500 error response from the yarn registry:
https://github.com/argoproj/argo-rollouts/actions/runs/11224243767/job/31200593228#step:13:351

Probably it's a good idea to re-run it before merging just in case by someone who can trigger that test manually.

[sonarcloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud