Add reboot and re-test to Testflinger job for DSS validation (New) #1634
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #1634 +/- ##
==========================================
- Coverage 48.80% 48.75% -0.05%
==========================================
Files 369 369
Lines 40192 40191 -1
Branches 6783 6774 -9
==========================================
- Hits 19616 19596 -20
- Misses 19857 19889 +32
+ Partials 719 706 -13
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
motjuste
force-pushed
the
CHECKBOX-1669-dss-reboot-testing-using-testflinger
branch
from
2eda760 to
5a9b6ac
Compare
There was a problem hiding this comment.
I don't think this is a good approach. You are running the same test plan twice, without having any way of checking at what point each of these tests have been run.
I am not sure how you plan to store the results. Are they only kept in Testflinger, or stored somewhere else (e.g. C3)?
Moreover, do you really want to re-run the whole test plan after rebooting, or would a subset of jobs be enough?
I would encourage you to write a dedicated test plan that includes a reboot job (this is a noreturn job, see Checkbox doc), with separate jobs depending on it run afterwards (see siblings in Checkbox documentation) to check that DSS is still alive and respond as expected to commands.
Description
The updates are only to the job definition template used by the GitHub workflow for running the DSS validations on machines from Testflinger. We reboot the machine and re-run the validations, with some delay in the middle to let things settle down.
Resolved issues
Documentation
There are no changes to the documentation.
Tests
The relevant DSS workflow run: https://github.com/canonical/checkbox/actions/runs/12143951564
to save ourselves some time...
The job linked above was actually run on a different branch, which consists of commits from this PR, and PR #1633 together. This is because the workflow runs can take a very long time, and while their changes don't overlap, the two PRs are worth testing together.
The failures on testing DSS latest/edge are expected due to a reported issue and do not stem from these changes to Checkbox. The one failure testing DSS latest/stable is during provisioning from Testflinger, before the Checkbox validations are even installed.