feature: add a generic slurm cluster submit option #283
Commits on May 2, 2024
-
-
-
Fix Slurm time format for workflows > 24 hours (timedelta)
- Incorrect `timedelta` handling caused Slurm job submission failures for workflows exceeding 24 hours. Python's `timedelta` format (e.g., "1 day, 1:00:00") was incompatible with Slurm. - Introduced `formatted_time` to convert `timedelta` objects to Slurm's required format (https://slurm.schedmd.com/sbatch.html#OPT_time). - This fix resolves errors caused by the incompatible time format in Slurm job submissions. - Applies the same time format fix for Slurm compatibility as commit af39a8d
-
Capture name of slurm partition from config
Users specify the name of the Slurm parition (queue) within the config.yml file in the directory they submit from. The queue name provided will be the slurm parition that all non-local jobs are submitted to.
-
Require slurm_partition key for
--slurmsubmit- Requires users of `--slurm` to specify the slurm partition in their config.yml file. - Removed the option to specify the slurm partition via the `--queue` when directly submitting with `--slurm`. - Renamed `--slurm-generic` flag to the simpler `--slurm`.
-
Improve Error Message for Missing
slurm_partitionThis commit improves the error message displayed when a user runs the `--slurm` option without specifying the required `slurm_partition` key-value pair in their `config.yml` file. The changes include: * Clearer error message highlighting the missing key-value pair and its purpose. * Color-coded output (if terminal supports) for better visual distinction between error and solution. These improvements make it easier for users to understand the error and take corrective action.
-
Add --slurm-partition option for
cgr configUsers can now specify the desired Slurm partition in their config.yml when building with `cgr config --slurm-partition <partition_name>`. This partition is used for jobs submitted with `cgr submit --slurm`.
-
docs: update conf.py with author information
Add RTI International contributors to the author list.
-
docs: include documentation about new
--slurmoption- Add documentation about the `--slurm-partition` option for the `cgr config` command. - Add documentation about the `--slurm` option for the `cgr submit` command. - Fix typo in a warning message about sample sheet requirement in `cgr_gwas_qc/cli/config.py` `docs/getting_started/running_pipeline.rst` file. - Fix typo in a message to Biowulf users in the `docs/getting_started/running_pipeline.rst` file. -
docs: update software params to align with current defaults
In commit bd74c72 we increased the `ibd_pi_hat_min` from 0.05 to 0.12 to focus on closer relatives. This commit aligns our documentation with that new default.
-
Update tests to include the generic slurm submission feature
We have the option to submit to a generic slurm cluster profile now with `cgr submit --slurm`, so this commit updates the tests to incorporate this new feature.
-
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.