Workflow test generator

[ilveroluca]

Write a workflow test generator:

• would have to be specific to particular workflow engine
• a test generator for Galaxy workflows seems feasible. Other workflow engines?
• it would work for workflows that "start" with tools that have tests in the tool shed
• procedure:
  lead_tools = { t | $t \in W$ and t processes workflow input data }

If all lead_tools:

1. are in tool shed
2. contain tests in tool shed then

• collect input datasets from tool tests
• Iterate over tool input datasets (and their combinations, if more than one tool)
  • execute workflow with tool test inputs
  • if workflow executes successfully, collect output as expected output to form a workflow test
• Result should be a set of workflow tests

The tests can be presented to workflow owner for approval.
If approved:

• generate new Workflow Testing RO-crate by extending the original one with the addition of the new tests
• provide it for download.

We couldn't at this stage adopt the new WTRO-crate directly because we're not yet equipped to execute tests.
Thus, user would have to

• extract tests from RO-crate
• Implement test execution somewhere, somehow
• Register new updated RO-crate that tells LM how to monitor those tests

This could also be implemented in an external tool (e.g., the LM cli) and later semi-automated through the maintenance plugin concept (#103).