Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Welling/run background subtraction parallel control rebased #719

Open
wants to merge 5 commits into
base: devel
Choose a base branch
from

Conversation

jswelling
Copy link
Collaborator

This branch cannot be deployed until a version of codex-pipeline downstream of both a4c9d72 and 4f78897 has been deployed, since it invokes a command line argument added by those mods. This branch adds a parameter to the invocation of codex-pipeline:run_background_subtraction.cwl in the codex_cytokit.py DAG which limits the amount of parallelism used by the Docker container in that step. Previously parallelism was not specified so it defaulted to the number of cores on the node, which is not what we want.

This change does limit the parallelism in run_background_subtraction, which is the last unlimited bit of the DAG's invocation of the codex-pipeline repo. It makes very little difference to running time in practice. The parallelism of the SPRM repo is still unlimited, so the codex_cytokit DAG still presents parallelism problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants