qa-test

[bjoernQ]

Thank you for your contribution!

We appreciate the time and effort you've put into this pull request.
To help us review it efficiently, please ensure you've gone through the following checklist:

Submission Checklist 📝

• I have updated existing examples or added new ones (if applicable).
• I have used cargo xtask fmt-packages command to ensure that all changed code is formatted correctly.
• My changes were added to the CHANGELOG.md in the proper section.
• I have added necessary changes to user code to the Migration Guide.
• My changes are in accordance to the esp-rs API guidelines

Extra:

• I have read the CONTRIBUTING.md guide and followed its instructions.

Pull Request Details 📖

Description

Closes #2566

We have the qa-test folder but running those tests was annoying and error-prone.

While it was always possible to run cargo xtask run-example FOLDER TARGET (i.e. w/o the example name) it always gave an error.

Now it will

• run all examples / tests for the given chip
• show the top-level comment - give the user time to attach / prepare whatever is needed
• let the user decide to run or skip the example (e.g. if the user doesn't have the needed external components at hand)
• run next example when the example was quit via CTRL-C

Additionally, this introduces a new key TAG which is used to group the examples (i.e. multiple examples needing the same wiring should run after each other)

In future we will move everything from examples over to qa-test (and adapt those examples to better serve as tests) but that is out of scope for this PR.

skip-changelog since this isn't really worth mentioning there

Testing

See above

[SergioGasquez]

LGTM, thanks! Also did some testing and now its way more convenient to test all the qa tests! Maybe we should thing if we require any other tag(s), maybe a tag for those qa-tests that dont require any extra-stuff is helpful

[MabezDev]

This is a nice improvement, especially for sequentially running qa-tests. I'm not sure I see the purpose of the tag system, how do you plan to use this in the future? If it's only going to be used in qa-test, which is all manual anyways, I don't think the tag system makes sense?

[bjoernQ]

This is a nice improvement, especially for sequentially running qa-tests. I'm not sure I see the purpose of the tag system, how do you plan to use this in the future? If it's only going to be used in qa-test, which is all manual anyways, I don't think the tag system makes sense?

The tags are used to order the examples - i.e. when multiple examples require the same wiring (e.g. a SPI-flash chip) they use the same tag and the user doesn't need to rewire multiple times

[MabezDev]

The tags are used to order the examples - i.e. when multiple examples require the same wiring (e.g. a SPI-flash chip) they use the same tag and the user doesn't need to rewire multiple times

Ah I see, that makes sense!