Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Threads documentation #288

Merged
merged 51 commits into from
Dec 4, 2024
Merged
Show file tree
Hide file tree
Changes from 31 commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
9313fd7
Threads tasks
giuliaelgarcia Jun 20, 2024
1c60c5e
Update index.rst
giuliaelgarcia Jun 20, 2024
b939250
Update pipeline_clustering_yml.md
giuliaelgarcia Jun 20, 2024
13836f1
Update pipeline_ingestion_yml.md
giuliaelgarcia Jun 20, 2024
6886b2a
Update pipeline_integration_yml.md
giuliaelgarcia Jun 20, 2024
e46b166
Update pipeline_preprocess_yml.md
giuliaelgarcia Jun 20, 2024
65e38cf
Update pipeline_refmap_yml.md
giuliaelgarcia Jun 20, 2024
765fef9
Update pipeline_visualization_yml.md
giuliaelgarcia Jun 20, 2024
d68f5ee
Update spatial_deconvolution.md
giuliaelgarcia Jun 20, 2024
c005711
Update spatial_preprocess.md
giuliaelgarcia Jun 20, 2024
9c6b886
Update spatial_qc.md
giuliaelgarcia Jun 20, 2024
c4fc55c
Delete docs/yaml_docs/threads_tasks_panpipes.xlsx
giuliaelgarcia Jun 20, 2024
7310166
Threads documentation
giuliaelgarcia Jun 20, 2024
9cb92e9
Update pipeline_clustering_yml.md
giuliaelgarcia Jun 20, 2024
b76b149
Update pipeline_ingestion_yml.md
giuliaelgarcia Jun 20, 2024
582db34
Update pipeline_integration_yml.md
giuliaelgarcia Jun 20, 2024
866d12a
Update pipeline_preprocess_yml.md
giuliaelgarcia Jun 20, 2024
6ca4530
Update pipeline_refmap_yml.md
giuliaelgarcia Jun 20, 2024
2c991f4
Update pipeline_visualization_yml.md
giuliaelgarcia Jun 20, 2024
95ce2ef
Update spatial_deconvolution.md
giuliaelgarcia Jun 20, 2024
3971ab2
Update spatial_preprocess.md
giuliaelgarcia Jun 20, 2024
28ca7ea
Update spatial_qc.md
giuliaelgarcia Jun 20, 2024
bfac191
Update pipeline_clustering_yml.md
giuliaelgarcia Jun 20, 2024
e7a2c40
Update pipeline_ingestion_yml.md
giuliaelgarcia Jun 20, 2024
9c9c83a
Update pipeline_integration_yml.md
giuliaelgarcia Jun 20, 2024
9f2e040
Update pipeline_preprocess_yml.md
giuliaelgarcia Jun 20, 2024
acf4835
Update pipeline_refmap_yml.md
giuliaelgarcia Jun 20, 2024
2e06eea
Update pipeline_visualization_yml.md
giuliaelgarcia Jun 20, 2024
27945f5
Update spatial_deconvolution.md
giuliaelgarcia Jun 20, 2024
878277e
Update threads_tasks_panpipes.csv
giuliaelgarcia Oct 15, 2024
e0b2a21
Updated to an HTML table
giuliaelgarcia Oct 15, 2024
76ba87f
Update pipeline_clustering_yml.md
giuliaelgarcia Nov 22, 2024
e7e33ed
Update pipeline_ingestion_yml.md
giuliaelgarcia Nov 22, 2024
8c56052
Update pipeline_integration_yml.md
giuliaelgarcia Nov 22, 2024
8af5044
Update pipeline_preprocess_yml.md
giuliaelgarcia Nov 22, 2024
080e513
Update pipeline_refmap_yml.md
giuliaelgarcia Nov 22, 2024
b9595c1
Update pipeline_refmap_yml.md
giuliaelgarcia Nov 22, 2024
1801d8d
Update pipeline_preprocess_yml.md
giuliaelgarcia Nov 22, 2024
527f586
Update pipeline_preprocess_yml.md
giuliaelgarcia Nov 22, 2024
f0a5e81
Update pipeline_integration_yml.md
giuliaelgarcia Nov 22, 2024
5c4fb23
Update pipeline_ingestion_yml.md
giuliaelgarcia Nov 22, 2024
80c1f87
Update pipeline_visualization_yml.md
giuliaelgarcia Nov 22, 2024
579349b
Update spatial_deconvolution.md
giuliaelgarcia Nov 22, 2024
8b7ad9c
Update spatial_preprocess.md
giuliaelgarcia Nov 22, 2024
606026e
Update spatial_qc.md
giuliaelgarcia Nov 22, 2024
b43a838
Update threads_tasks_panpipes.md
bio-la Dec 4, 2024
5c6b5c4
fixed errors
bio-la Dec 4, 2024
5cbefaf
fixed path to table
bio-la Dec 4, 2024
771a7ee
added title
bio-la Dec 4, 2024
4caff39
paths
bio-la Dec 4, 2024
004eb5b
space
bio-la Dec 4, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/yaml_docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,5 +14,6 @@ Workflows configuration files
spatial_deconvolution
pipeline_visualization_yml
pipeline_refmap_yml
threads_tasks_panpipes


2 changes: 1 addition & 1 deletion docs/yaml_docs/pipeline_clustering_yml.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ You can download the different clustering pipeline.yml files here:
## Compute resources options

- <span class="parameter">resources</span><br>
Computing resources to use, specifically the number of threads used for parallel jobs.
Computing resources to use, specifically the number of threads used for parallel jobs, check [threads_tasks_panpipes.csv](https://github.com/DendrouLab/panpipes/blob/threads_doc_g/docs/yaml_docs/threads_tasks_panpipes.csv) for more information on which threads each specific task requires.
Specified by the following three parameters:
- <span class="parameter">threads_high</span> `Integer`, Default: 2<br>
Number of threads used for high intensity computing tasks.
Expand Down
2 changes: 1 addition & 1 deletion docs/yaml_docs/pipeline_ingestion_yml.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ You can download the different ingestion `pipeline.yml` files here:
## Compute resources options

<span class="parameter">resources</span><br>
Computing resources to use, specifically the number of threads used for parallel jobs.
Computing resources to use, specifically the number of threads used for parallel jobs, check [threads_tasks_panpipes.csv](https://github.com/DendrouLab/panpipes/blob/threads_doc_g/docs/yaml_docs/threads_tasks_panpipes.csv) for more information on which threads each specific task requires.
Specified by the following three parameters:
- <span class="parameter">threads_high</span> `Integer`, Default: 1<br>
Number of threads used for high intensity computing tasks.
Expand Down
2 changes: 1 addition & 1 deletion docs/yaml_docs/pipeline_integration_yml.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ For more information on functionalities implemented in `panpipes` to read the co
## Compute resources options

<span class="parameter">resources</span><br>
Computing resources to use, specifically the number of threads used for parallel jobs.
Computing resources to use, specifically the number of threads used for parallel jobs. Check [threads_tasks_panpipes.csv](https://github.com/DendrouLab/panpipes/blob/threads_doc_g/docs/yaml_docs/threads_tasks_panpipes.csv) for more information on which threads each specific task requires.
Specified by the following parameters:
- <span class="parameter">threads_high</span> `Integer`, Default: 1<br>
Number of threads used for high intensity computing tasks.
Expand Down
2 changes: 1 addition & 1 deletion docs/yaml_docs/pipeline_preprocess_yml.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ You can download the different preprocess `pipeline.yml` files here:
## Compute resources options

<span class="parameter">resources</span><br>
Computing resources to use, specifically the number of threads used for parallel jobs.
Computing resources to use, specifically the number of threads used for parallel jobs. Check [threads_tasks_panpipes.csv](https://github.com/DendrouLab/panpipes/blob/threads_doc_g/docs/yaml_docs/threads_tasks_panpipes.csv) for more information on which threads each specific task requires.
Specified by the following three parameters:
- <span class="parameter">threads_high</span> `Integer`, Default: 2<br>
Number of threads used for high intensity computing tasks.
Expand Down
2 changes: 1 addition & 1 deletion docs/yaml_docs/pipeline_refmap_yml.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ You can download the different refmap `pipeline.yml` files here:
## Compute resources options

<span class="parameter">resources</span><br>
Computing resources to use, specifically the number of threads used for parallel jobs.
Computing resources to use, specifically the number of threads used for parallel jobs. Check [threads_tasks_panpipes.csv](https://github.com/DendrouLab/panpipes/blob/threads_doc_g/docs/yaml_docs/threads_tasks_panpipes.csv) for more information on which threads each specific task requires.
Specified by the following three parameters:
- <span class="parameter">threads_high</span> `Integer`, Default: 1<br>
Number of threads used for high intensity computing tasks.
Expand Down
2 changes: 1 addition & 1 deletion docs/yaml_docs/pipeline_visualization_yml.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ You can download the different ingestion `pipeline.yml` files here:

## Compute resources options
<span class="parameter">resources</span><br>
Computing resources to use, specifically the number of threads used for parallel jobs.
Computing resources to use, specifically the number of threads used for parallel jobs. Check [threads_tasks_panpipes.csv](https://github.com/DendrouLab/panpipes/blob/threads_doc_g/docs/yaml_docs/threads_tasks_panpipes.csv) for more information on which threads each specific task requires.
Specified by the following three parameters:
- <span class="parameter">threads_high</span> `Integer`, Default: 1<br>
Number of threads used for high intensity computing tasks.
Expand Down
2 changes: 1 addition & 1 deletion docs/yaml_docs/spatial_deconvolution.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ For more information on functionalities implemented in `panpipes` to read the co
## 0. Compute Resource Options

<span class="parameter">resources</span><br>
Computing resources to use, specifically the number of threads used for parallel jobs.
Computing resources to use, specifically the number of threads used for parallel jobs. Check [threads_tasks_panpipes.csv](https://github.com/DendrouLab/panpipes/blob/threads_doc_g/docs/yaml_docs/threads_tasks_panpipes.csv) for more information on which threads each specific task requires.
Specified by the following three parameters:
- <span class="parameter">threads_high</span> `Integer`, Default: 1<br>
Number of threads used for high intensity computing tasks.
Expand Down
2 changes: 1 addition & 1 deletion docs/yaml_docs/spatial_preprocess.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ You can download the different preprocess pipeline.yml files here:
## 0. Compute Resource Options

<span class="parameter">resources</span><br>
Computing resources to use, specifically the number of threads used for parallel jobs.
Computing resources to use, specifically the number of threads used for parallel jobs. Check [threads_tasks_panpipes.csv](https://github.com/DendrouLab/panpipes/blob/threads_doc_g/docs/yaml_docs/threads_tasks_panpipes.csv) for more information on which threads each specific task requires.
Specified by the following three parameters:
- <span class="parameter">threads_high</span> `Integer`, Default: 1<br>
Number of threads used for high intensity computing tasks.
Expand Down
2 changes: 1 addition & 1 deletion docs/yaml_docs/spatial_qc.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ For more information on functionalities implemented in `panpipes` to read the co


<span class="parameter">resources</span><br>
Computing resources to use, specifically the number of threads used for parallel jobs.
Computing resources to use, specifically the number of threads used for parallel jobs. check [threads_tasks_panpipes.csv](https://github.com/DendrouLab/panpipes/blob/threads_doc_g/docs/yaml_docs/threads_tasks_panpipes.csv) for more information on which threads each specific task requires.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@giuliaelgarcia you deleted the csv file in favour of the md file but you're still referencing the csv in all the documentation pages. can you please fix?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bio-la Just fixed this!

Specified by the following three parameters:
- <span class="parameter">threads_high</span> `Integer`, Default: 1<br>
Number of threads used for high intensity computing tasks.
Expand Down
268 changes: 268 additions & 0 deletions docs/yaml_docs/threads_tasks_panpipes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,268 @@
<table>
<tr>
<th colspan="3">Task ingest</th>
</tr>
<tr>
<th>threads_high</th>
<th>threads_medium</th>
<th>threads_low</th>
</tr>
<tr>
<td>Creating h5mu from filtered data files</td>
<td>load_mudatas</td>
<td>run_repertoire_qc</td>
</tr>
<tr>
<td>Creating h5mu from bg data files</td>
<td>load_bg_mudatas</td>
<td>run_atac_qc</td>
</tr>
<tr>
<td>rna QC</td>
<td>downsample_bg_mudatas</td>
<td>plot_qc</td>
</tr>
<tr>
<td>prot QC</td>
<td>run_scrublet</td>
<td>10X metrics plotting</td>
</tr>
<tr>
<td>prot QC</td>
<td></td>
<td></td>
</tr>
<tr>
<th colspan="3">Task preprocessed</th>
</tr>
<tr>
<th>threads_high</th>
<th></th>
<th></th>
</tr>
<tr>
<td>assess background</td>
<td></td>
<td>threads_low</td>
</tr>
<tr>
<td>rna_preprocess</td>
<td></td>
<td>filter_mudata</td>
</tr>
<tr>
<td>prot_preprocess</td>
<td></td>
<td>postfilterplot</td>
</tr>
<tr>
<td>atac_preprocess</td>
<td></td>
<td>downsample</td>
</tr>
<tr>
<th colspan="3">Task integration</th>
</tr>
<tr>
<th>threads_high</th>
<th>threads_medium</th>
<th>threads_low</th>
</tr>
<tr>
<td>run_no_batch_correct_rna</td>
<td>Evaluation</td>
<td>run_lisi</td>
</tr>
<tr>
<td>run_bbknn_rna</td>
<td>plot_umaps</td>
<td></td>
</tr>
<tr>
<td>run_harmony_rna</td>
<td>run_scib_metrics</td>
<td></td>
</tr>
<tr>
<td>run_combat_rna</td>
<td></td>
<td></td>
</tr>
<tr>
<td>run_scanorama_rna</td>
<td></td>
<td></td>
</tr>
<tr>
<td>run_scvi_rna</td>
<td></td>
<td></td>
</tr>
<tr>
<td>run_no_batch_correct_prot</td>
<td></td>
<td></td>
</tr>
<tr>
<td>run_harmony_prot</td>
<td></td>
<td></td>
</tr>
<tr>
<td>run_bbknn_prot</td>
<td></td>
<td></td>
</tr>
<tr>
<td>run_combat_prot</td>
<td></td>
<td></td>
</tr>
<tr>
<td>run_no_batch_correct_atac</td>
<td></td>
<td></td>
</tr>
<tr>
<td>run_harmony_atac</td>
<td></td>
<td></td>
</tr>
<tr>
<td>run_bbknn_atac</td>
<td></td>
<td></td>
</tr>
<tr>
<td>run_totalvi</td>
<td></td>
<td></td>
</tr>
<tr>
<td>run_multivi</td>
<td></td>
<td></td>
</tr>
<tr>
<td>run_mofa</td>
<td></td>
<td></td>
</tr>
<tr>
<td>run_wnn</td>
<td></td>
<td></td>
</tr>
<tr>
<td>merge_integration</td>
<td></td>
<td></td>
</tr>
<tr>
<th colspan="3">Task clustering</th>
</tr>
<tr>
<th>threads_high</th>
<th>threads_medium</th>
<th>threads_low</th>
</tr>
<tr>
<td>run_neighbors</td>
<td>run_clustering</td>
<td>clustering</td>
</tr>
<tr>
<td>run_umap</td>
<td>collate_mdata</td>
<td>clustering</td>
</tr>
<tr>
<td>find_markers</td>
<td>plot_cluster_umaps</td>
<td></td>
</tr>
<tr>
<td></td>
<td>plot_markers</td>
<td></td>
</tr>
<tr>
<th colspan="3">Task vis</th>
</tr>
<tr>
<th>threads_high</th>
<th></th>
<th>threads_low</th>
</tr>
<tr>
<td>plot_custom_markers_per_group</td>
<td></td>
<td>plot_metrics</td>
</tr>
<tr>
<td>plot_custom_markers_umap</td>
<td></td>
<td></td>
</tr>
<tr>
<td>plot_categorical_umaps</td>
<td></td>
<td></td>
</tr>
<tr>
<td>write_obs</td>
<td></td>
<td></td>
</tr>
<tr>
<td>plot_scatters</td>
<td></td>
<td></td>
</tr>
<tr>
<td>Task refmap</td>
<td></td>
<td></td>
</tr>
<tr>
<td>threads_high</td>
<td></td>
<td></td>
</tr>
<tr>
<td>run_refmap_scvi</td>
<td></td>
<td></td>
</tr>
<tr>
<td>run_scib_refmap</td>
<td></td>
<td></td>
</tr>
<tr>
<th colspan="3">Task preprocessed spatial</th>
</tr>
<tr>
<th>threads_high</th>
<th></th>
<th>threads_low</th>
</tr>
<tr>
<td>spatial_preprocess</td>
<td></td>
<td>filter_mudata</td>
</tr>
<tr>
<th colspan="3">Task Spatial</th>
</tr>
<tr>
<th>threads_high</th>
<th></th>
<th>threads_low</th>
</tr>
<tr>
<td>load_mudata</td>
<td></td>
<td>plotQC_spatial</td>
</tr>
</table>
Loading