Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[24.0] Record missing outputs as test errors #17874

Merged

Conversation

bernt-matthias
Copy link
Contributor

Xref #16094

How to test the changes?

(Select all options that apply)

  • I've included appropriate automated tests.
  • This is a refactoring of components with existing test coverage.
  • Instructions for manual testing are as follows:
    1. [add testing steps and prerequisites here if you didn't write automated tests covering all your changes]

License

  • I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

bernt-matthias and others added 12 commits April 2, 2024 09:46
- remove the output that is always present
- add a test that has no outputs
from planemo the verify_tool function is called from
[within an try-except block](https://github.com/galaxyproject/planemo/blob/1aa3eb05a97ad20c0be6f6560ab5cec090e76612/planemo/engine/galaxy.py#L109)
which silently catches any exception.

Thus any exception raised from within verify_tool will not be
detected, i.e. the assertion needs to be moved into
`_verify_outputs` (which also seems to make sense by name)
in order to make verify_tool record the problem properly.
thereby we implicitly make the assumption that a test needs to produce
an output

also make message more specific
otherwise the tool test will just return a list out of bounds exception
which is unclear to the user
Co-authored-by: Marius van den Beek <m.vandenbeek@gmail.com>
then it might be clearer what's going on here
@github-actions github-actions bot added this to the 24.1 milestone Apr 2, 2024
@bernt-matthias bernt-matthias modified the milestones: 24.1, 24.0 Apr 2, 2024
@mvdbeek mvdbeek changed the title [24.0] Missing outputs should be recorded as test errors [24.0] Record missing outputs as test errors Apr 2, 2024
@mvdbeek mvdbeek merged commit e331ccb into galaxyproject:release_24.0 Apr 2, 2024
50 of 52 checks passed
@bernt-matthias bernt-matthias deleted the assert-empty-output_24.0 branch April 2, 2024 21:00
mvdbeek added a commit to mvdbeek/galaxy that referenced this pull request Apr 3, 2024
That fixes test_optional_workflow_output, which started failing after
galaxyproject#17874, which removed the
last static output of `output_filter`, and so there would never be any
active datasets to wait on.
nsoranzo added a commit to mvdbeek/galaxy that referenced this pull request Apr 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants