Sfitz add mosdepth quantize

.gitignore

@@ -81,7 +81,3 @@ work/
 *.tar
 *.zip
 
-# Other
-test/*
-test/*/*
-slurm-*.out

CHANGELOG.md

@@ -10,6 +10,7 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
 ## [Unreleased]
 
 ### Added
+- Add `mosdepth` quantize workflow
 - Add `mosdepth` coverage windows workflow
 - Add `FastQC` workflow
 - Add per readgroup and per library functionality

README.md

@@ -13,7 +13,7 @@
   9. [References](#references)
 ## Overview
 
-This pipeline takes BAMs and runs selected Quality Control (QC) steps. Available algorithms are currently `SAMtools stats`, `Picard CollectWgsMetrics` and `Qualimap bamqc`. Generally either `Qualimap bamqc` or `SAMtools stats and Picard CollectWgsMetrics` should be run, not both. `Qualimap bamqc` uses a lot of memory and should not be run within `uclahs-cds/metapipeline-DNA`. Input can include any combination of tumor and normal BAMs from a single donor. Each will be processed independently. RNA specific QC is not yet implemented but is expected soon.
+This pipeline takes BAMs and runs selected Quality Control (QC) steps. Available algorithms are currently `SAMtools stats`, `Picard CollectWgsMetrics`, `FastQC`, `Qualimap bamqc`, `mosdepth coverage` and `mosdepth quantize`. Generally either `Qualimap bamqc` or `SAMtools stats and Picard CollectWgsMetrics` should be run, not both. `Qualimap bamqc` uses a lot of memory and should not be run within `uclahs-cds/metapipeline-DNA`. Input can include any combination of tumor and normal BAMs from a single donor. Each will be processed independently. RNA specific QC is not yet implemented but is expected soon.
 
 ---
 
@@ -51,9 +51,12 @@ Each of the below algorithms, if selected, will run in parallel subject to avail
 ### 4. FastQC
 [FastQC](https://www.bioinformatics.babraham.ac.uk/projects/fastqc/Help/) aims to provide a QC report which can spot problems which originate either in the sequencer or in the starting library material.
 
-### 5. mosdepth windows
+### 5. mosdepth coverage
 [mosdepth](https://github.com/brentp/mosdepth) by windows provides fast BAM/CRAM depth calculation.
 
+### 6. mosdepth quantize
+[mosdepth](https://github.com/brentp/mosdepth) quantize creates a bed file labeling regions within specified coverage thresholds. Similar to GATK's callable loci tool.
+
 ---
 
 ## Inputs
@@ -78,7 +81,7 @@ input:
 
 | Field | Type | Required | Description |
 | ----- | ---- | ------------ | ------------------------ |
-| `algorithm` | list | no | List of tools to be run: ['fastqc', 'samtools_stats', 'collectwgsmetrics', 'mosdepth_coverage', 'qualimap_bamqc'], default = ['stats', 'collectwgsmetrics'] |
+| `algorithm` | list | no | List of tools to be run: ['fastqc', 'samtools_stats', 'collectwgsmetrics', 'mosdepth_coverage', 'mosdepth_quantize', 'qualimap_bamqc'], default = ['stats', 'collectwgsmetrics'] |
 | `reference` | path | yes/no | Reference fasta is required only for `CollectWgsMetrics` |
 | `output_dir` | path | yes | Not required if `blcds_registered_dataset` = `true` |
 | `blcds_registered_dataset` | boolean | no | Default is `false`. Only `uclahs_cds` users should change this. When `true`, BLCDS folder structure is used |
@@ -92,12 +95,6 @@ input:
 | stats_remove_duplicates | boolean | no | Ignore reads marked as duplicate. Default = `false` |
 | stats_additional_options | string | no | Any additional options recognized by `samtools stats` |
 
-#### FastQC specific configuration
-| Field | Type | Required | Description |
-| ----- | ---- | ------------ | ------------------------ |
-| fastqc_level | string | yes | 'readgroup', 'library' or 'sample' |
-| fastqc_additional_options | string | no | Any additional options recognized by `FastQC` |
-
 #### Picard specific configuration
 | Field | Type | Required | Description |
 | ----- | ---- | ------------ | ------------------------ |
@@ -107,19 +104,37 @@ input:
 | cwm_use_fast_algorithm | boolean | no | If `true`, fast algorithm is used |
 | cwm_additional_options | string | no | Any additional options recognized by `CollectWgsMetrics` |
 
-#### mosdepth windows specific configuration
+#### FastQC specific configuration
 | Field | Type | Required | Description |
 | ----- | ---- | ------------ | ------------------------ |
-| mosdepth_use_fast_algorithm | boolean | no | `fast` algorithm ignores read pair overlaps and CIGARs. It should not be used on libraries with small insert sizes. Default = `false` |
-| mosdepth_window_size | integer | no | Size for `mosdepth windows` coverage calculations |
-| mosdepth_additional_options | string | no | Any additional options recognized by `mosdepth` |
+| fastqc_level | string | yes | 'readgroup', 'library' or 'sample' |
+| fastqc_additional_options | string | no | Any additional options recognized by `FastQC` |
 
 #### Qualimap specific configuration
 | Field | Type | Required | Description |
 | ----- | ---- | ------------ | ------------------------ |
 | bamqc_output_format | string | no | Choice of 'pdf' or 'html', default = 'pdf' |
 | bamqc_additional_options | string | no | Any additional options recognized by `bamqc` |
 
+#### mosdepth coverage specific configuration
+| Field | Type | Required | Description |
+| ----- | ---- | ------------ | ------------------------ |
+| mosdepth_use_fast_algorithm | boolean | no | `fast` algorithm ignores read pair overlaps and CIGARs. It should not be used on libraries with small insert sizes. Default = `false` |
+| mosdepth_per_base_output | boolean | no | Output coverage for every base. Default = `true` |
+| mosdepth_window_size | integer | no | Size for `mosdepth windows` coverage calculations |
+| mosdepth_additional_options | string | no | Any additional options recognized by `mosdepth`, `--mapq 20 recommended` |
+
+#### mosdepth quantize specific configuration
+| Field | Type | Required | Description |
+| ----- | ---- | ------------ | ------------------------ |
+| mosdepth_quantize_cutoffs | string | no | cutoffs for coverage regions. Default = `0:1:5:150` |
+| mosdepth_quantize_use_fast_algorithm | boolean | no | `fast` algorithm ignores read pair overlaps and CIGARs. It should not be used on libraries with small insert sizes. Default = `false` |
+| mosdepth_q0_label | string | no | lowest coverage regions label. Default = `Q0`
+| mosdepth_q1_label | string | no | next coverage regions label. Default = `Q1`
+| mosdepth_q2_label | string | no | next coverage regions label. Default = `Q2`
+| mosdepth_q3_label | string | no | highest coverage regions label. Default = `Q3`
+| mosdepth_quantize_additional_options | string | no | Any additional options recognized by `mosdepth`. `--mapq 20 recommended` |
+
 #### Base resource allocation updaters
 To update the base resource (cpus or memory) allocations for processes, use the following structure. The default allocations can be found in the [node-specific config files](./config/)
 ```Nextflow
@@ -172,14 +187,21 @@ base_resource_update {
 
 | Output | Description |
 | ------------ | ------------------------ |
-| `{SAMtools-version}_{dataset_id}_{sample_id}_stats.txt` | SAMtools stats results |
-| `{Picard-version}_{dataset_id}_{sample_id}_wgs-metrics.txt` | Picard CollectWgsMetrics results |
-| `{Qualimap-version}_{dataset_id}_{sample_id}_stats` | Directory of Qualimap results, including, `genome_results.txt` and either `.pdf` or `.html and supporting directories`|
-| `{FastQC-version}_{dataset_id}_{sample_id}_fastqc` | Directory of FastQC results |
-| `{mosdepth-version}_{dataset_id}_{sample_id}-{window_size}.mosdepth.summary.txt` | Coverage by region with a final line for `total` |
-| `{mosdepth-version}_{dataset_id}_{sample_id}-{window_size}.mosdepth.global.dist.txt` | a cumulative distribution indicating the proportion of total bases that were covered for at least a given coverage value |
-| `{mosdepth-version}_{dataset_id}_{sample_id}-{window_size}.mosdepth.region.dist.txt` | a cumulative distribution indicating the proportion of the windows that were covered for at least a given coverage value |
-| `{mosdepth-version}_{dataset_id}_{sample_id}-{window_size}.regions.bed.gz` | Bedfile giving coverage for each window |
+| `{SAMtools-version}_{dataset_id}_{sample_id}_stats.txt` | `SAMtools stats` sample level results |
+| `{SAMtools-version}_{dataset_id}_{sample_id}-{library_id}_stats.txt` | `SAMtools stats` library level results |
+| `{SAMtools-version}_{dataset_id}_{sample_id}-{library_id}-{rg_id}_stats.txt` | `SAMtools stats` readgroup level results |
+| `{Picard-version}_{dataset_id}_{sample_id}_wgs-metrics.txt` | `Picard CollectWgsMetrics` results |
+| `{Qualimap-version}_{dataset_id}_{sample_id}_stats` | Directory of `Qualimap` results, including, `genome_results.txt` and either `.pdf` or `.html and supporting directories`|
+| `{FastQC-version}_{dataset_id}_{sample_id}_fastqc` | Directory of sample level `FastQC` results |
+| `{FastQC-version}_{dataset_id}_{sample_id}-{library_id}_fastqc` | Directory of library level `FastQC` results |
+| `{FastQC-version}_{dataset_id}_{sample_id}-{library_id}-{rg_id}_fastqc` | Directory of readgroup level `FastQC` results |
+| `{mosdepth-version}_{dataset_id}_{sample_id}-{window_size}.mosdepth.summary.txt` | `mosdepth` coverage results by region with a final line for `total` |
+| `{mosdepth-version}_{dataset_id}_{sample_id}-{window_size}.mosdepth.global.dist.txt` | `mosdepth` coverage cumulative distribution indicating the proportion of total bases that were covered for at least a given coverage value |
+| `{mosdepth-version}_{dataset_id}_{sample_id}-{window_size}.mosdepth.region.dist.txt` | `mosdepth` coverage cumulative distribution indicating the proportion of the windows that were covered for at least a given coverage value |
+| `{mosdepth-version}_{dataset_id}_{sample_id}-{window_size}.regions.bed.gz` | `mosdepth` coverage bedfile giving coverage for each window |
+| `{mosdepth-version}_{dataset_id}_{sample_id}-quantize-{q0}-{q1}-{q2}-{q3}.mosdepth.summary.txt` | `mosdepth` quantize coverage results by region with a final line for `total` |
+| `{mosdepth-version}_{dataset_id}_{sample_id}-quantize-{q0}-{q1}-{q2}-{q3}.mosdepth.global.dist.txt` | `mosdepth` quantize cumulative distribution indicating the proportion of total bases that were covered for at least a given coverage value |
+| `{mosdepth-version}_{dataset_id}_{sample_id}-quantize-{q0}-{q1}-{q2}-{q3}.quantized.bed.gz` | `mosdepth` quantize bed file
 
 ---
 

config/F16.config

@@ -43,6 +43,16 @@ process {
             }
         }
     }
+    withName: quantize_coverage_mosdepth {
+        cpus = 1
+        memory = 8.GB
+        retry_strategy {
+            memory {
+                strategy = 'add'
+                operand = 4.GB
+            }
+        }
+    }
     withName: run_statsReadgroups_SAMtools {
         cpus = 1
         memory = 1.GB

config/F2.config

@@ -43,6 +43,26 @@ process {
             }
         }
     }
+    withName: assess_coverage_mosdepth {
+        cpus = 1
+        memory = 1500.MB
+        retry_strategy {
+            memory {
+                strategy = 'add'
+                operand = 2000.MB
+            }
+        }
+    }
+    withName: quantize_coverage_mosdepth {
+        cpus = 1
+        memory = 1500.MB
+        retry_strategy {
+            memory {
+                strategy = 'add'
+                operand = 2000.MB
+            }
+        }
+    }
     withName: run_statsReadgroups_SAMtools {
         cpus = 1
         memory = 1.5.GB

config/F32.config

@@ -43,6 +43,16 @@ process {
             }
         }
     }
+    withName: quantize_coverage_mosdepth {
+        cpus = 1
+        memory = 8.GB
+        retry_strategy {
+            memory {
+                strategy = 'add'
+                operand = 4.GB
+            }
+        }
+    }
     withName: run_statsReadgroups_SAMtools {
         cpus = 1
         memory = 1.GB

config/F4.config

@@ -43,6 +43,16 @@ process {
             }
         }
     }
+    withName: quantize_coverage_mosdepth {
+        cpus = 1
+        memory = 8.GB
+        retry_strategy {
+            memory {
+                strategy = 'add'
+                operand = 2.GB
+            }
+        }
+    }
     withName: run_statsReadgroups_SAMtools {
         cpus = 1
         memory = 1.GB

config/F72.config

@@ -43,6 +43,16 @@ process {
             }
         }
     }
+    withName: quantize_coverage_mosdepth {
+        cpus = 1
+        memory = 8.GB
+        retry_strategy {
+            memory {
+                strategy = 'add'
+                operand = 8.GB
+            }
+        }
+    }
     withName: run_statsReadgroups_SAMtools {
         cpus = 1
         memory = 1.GB

config/F8.config

@@ -43,6 +43,16 @@ process {
             }
         }
     }
+    withName: quantize_coverage_mosdepth {
+        cpus = 1
+        memory = 8.GB
+        retry_strategy {
+            memory {
+                strategy = 'add'
+                operand = 2.GB
+            }
+        }
+    }
     withName: run_statsReadgroups_SAMtools {
         cpus = 1
         memory = 1.GB

config/M64.config

@@ -43,6 +43,16 @@ process {
             }
         }
     }
+    withName: quantize_coverage_mosdepth {
+        cpus = 1
+        memory = 8.GB
+        retry_strategy {
+            memory {
+                strategy = 'add'
+                operand = 8.GB
+            }
+        }
+    }
     withName: run_statsReadgroups_SAMtools {
         cpus = 1
         memory = 1.GB

config/schema.yaml

@@ -9,11 +9,12 @@ dataset_id:
   help: 'Dataset identifier'
 algorithm:
   type: 'List'
-  required: false
+  required: true
   help: 'List of QC algorithms'
   choices:
     - fastqc
     - mosdepth_coverage
+    - mosdepth_quantize
     - samtools_stats
     - collectwgsmetrics
     - qualimap_bamqc
@@ -82,22 +83,58 @@ mosdepth_use_fast_algorithm:
   required: false
   default: true
   help: 'fast algorithm ignores read pair overlaps and CIGAR information'
-mosdepth_window_size:
-  type: 'Integer'
-  required: false
-  default: 500
-  help: 'Window size for mosdepth coverage calculation'
 mosdepth_per_base_output:
   type: 'Bool'
   required: false
   default: true
   help: 'Output per-base coverage'
+mosdepth_window_size:
+  type: 'Integer'
+  required: false
+  default: 500
+  help: 'Window size for mosdepth coverage calculation'
 mosdepth_additional_options:
   type: 'String'
   required: false
   allow_empty: true
   default: ''
   help: 'Additional arguments for mosdepth command'
+mosdepth_quantize_cutoffs:
+  type: 'String'
+  required: false
+  default: '0:1:5:150'
+  help: 'Quantize coverage values into these bins'
+mosdepth_quantize_use_fast_algorithm:
+  type: 'Bool'
+  required: false
+  default: false
+  help: 'Use fast algorithm for quantizing coverage values'
+mosdepth_q0_label:
+  type: 'String'
+  required: false
+  default: 'Q0'
+  help: 'Label for lowest coverage bin'
+mosdepth_q1_label:
+  type: 'String'
+  required: false
+  default: 'Q1'
+  help: 'Label for second lowest coverage bin'
+mosdepth_q2_label:
+  type: 'String'
+  required: false
+  default: 'Q2'
+  help: 'Label for third lowest coverage bin'
+mosdepth_q3_label:
+  type: 'String'
+  required: false
+  default: 'Q3'
+  help: 'Label for highest coverage bin'
+mosdepth_quantize_additional_options:
+  type: 'String'
+  required: false
+  allow_empty: true
+  default: ''
+  help: 'Additional arguments for mosdepth-quantize command'
 cwm_coverage_cap:
   type: 'Integer'
   required: false

config/template.config

@@ -8,7 +8,7 @@ includeConfig "${projectDir}/nextflow.config"
 
 // Inputs/parameters of the pipeline
 params {
-    algorithm = ['samtools_stats', 'collectwgsmetrics'] // 'fastqc', 'samtools_stats', 'collectwgsmetrics', 'mosdepth_coverage', 'qualimap_bamqc'
+    algorithm = ['samtools_stats', 'collectwgsmetrics'] // 'fastqc', 'samtools_stats', 'collectwgsmetrics', 'mosdepth_quantize', 'mosdepth_coverage', 'qualimap_bamqc'
     reference = '/hot/resource/reference-genome/GRCh38-BI-20160721/Homo_sapiens_assembly38.fasta'
     output_dir = '/path/to/output/directory'
     blcds_registered_dataset = false // if you want the output to be registered
@@ -22,9 +22,18 @@ params {
     // mosdepth window-base coverage options
     // fast algorithm ignores read pair overlaps and should not be used on libraries with small insert sizes
     mosdepth_use_fast_algorithm = false
-    mosdepth_window_size = 500
     mosdepth_per_base_output = true
-    mosdepth_additional_options = ''
+    mosdepth_window_size = 500
+    mosdepth_additional_options = '--mapq 20'
+
+    // mosdepth quantized coverage (like GATK's Callable Regions)
+    mosdepth_quantize_cutoffs = '0:1:5:150'
+    mosdepth_quantize_use_fast_algorithm = false
+    mosdepth_q0_label = 'NO_COVERAGE'
+    mosdepth_q1_label = 'LOW_COVERAGE'
+    mosdepth_q2_label = 'CALLABLE'
+    mosdepth_q3_label = 'HIGH_COVERAGE'
+    mosdepth_quantize_additional_options = '--mapq 20'
 
     // Picard CollectWgsMetrics options
     cwm_coverage_cap = 1000