Workflow Landing Requests

[jmchilton]

I've added a little CLI tool for generating workflow landing requests URLs. External clients won't need to use this but it is a simple example with an easy to follow format and flow that describes how to use the APIs and generate custom forms with pre-populated data for users.

$ . .venv/bin/activate; PYTHONPATH=lib python lib/galaxy/tool_util/client/landing.py -g http://localhost:8081 -s mycoolthing simple_workflow
Your customized form is located at http://localhost:8081/workflow_landings/0d3862e7-345f-4805-a080-5a2cbc2fc547?secret=mycoolthing

This pairs well with the new {src: "url",...} data input dictionaries and a potential minimal workflow running UI (pictured below).

The UI piece is clearly an MVP - hopefully a tool form UI expert can take it over.

How to test the changes?

(Select all options that apply)

• I've included appropriate automated tests.
• This is a refactoring of components with existing test coverage.
• Instructions for manual testing are as follows:
  1. [add testing steps and prerequisites here if you didn't write automated tests covering all your changes]

License

• I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

[jmchilton]

This is interwound with the tool request API that I thought was closer than it was. I'm finding so many corner cases in modeling tools. If this becomes a blocker for anyone I think I could pretty easily pull out the database migration, the workflow landing stuff, the workflow API that adds {src: "url", ... } inputs, and the form input for the URI parameters.

Alternatively I could pull just the tool API & backend enhancements out and push them back into #17393 and we could keep all the modeling enhancements in here and treat this a bit more like the CWL branch where we have models we sync up but keep the plumbing out until it is solid.

Otherwise I will just spend this week trying to work on the models and just spin some smaller stuff out here and there where it makes sense.

[jmchilton]

What's the problem you're hitting ?

I'm getting my butt kicked by a dozen little bits of complexity around how parameters work and then building and validating models for what job has as tool state. After a week I've decided that you cannot really infer a default for data column without a runtime... and then runtime that populates the defaults for data columns (populate) also replaces incoming state with HDAs for instance. So I am not getting that clean pipeline of parameters that I want... "fill in static defaults define structure -> fill in dynamic defaults -> record and validate state -> run populate on validated state -> create wrappers". For reproducibility I think it would be really great if we could preserve and validate the thing we hand off to populate and then to the wrappers. I've decided that is going to take some more restructuring though. The API works though - I think by the end of the week I was able to get all the framework tools to use the request API (or at least the ones with validating test cases) - I just think it is important that all the jobs that we have tool requests for also have validated job state for - I hope that makes sense. Functional is good but tracking and reproducible are important here.

I have ripped out the tool landing API, the tool request API, and the simplified workflow UI out of this PR and pushed them back upstream into #17393. However... the workflow landing stuff uses the data request objects from the tool state model upgrades for the request API so this has all of those updates and the workflow landing database stuff was written and tested with the tool request DB migration - so that migration and the corresponding DB models are included here. Ideally I would separate those two things out - but I don't think it is worth the effort - we know we're going to want all of that in the tool request API whenever that is ready and having those things in for now eases that development and improves the API we added to share the meta model in the 2.0 tool shed.

[bgruening]

I guess the description and the file name is wrong?

[bgruening]

I find the term library (here and elsewhere) a bit confusing. What about:

• controls
• properties
• options
• configurations
  ??

[jmchilton]

Is "catalog" okay? I think of those choices - options is the best but it is feels a little overloaded with options in the workflow instead of options for which workflow to run.

[bgruening]

catalog is also nice :)