Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add microbiome analysis workflows #182

Merged
merged 71 commits into from
Jun 25, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
8340c8a
adding microbiome analysis workflows to IWC with test data
EngyNasr Mar 7, 2023
56a6dbf
adding Changelog, REadme and dockstore yml file
EngyNasr Mar 9, 2023
e72aa56
solving linting issues
EngyNasr Mar 19, 2023
a401cba
solving linting error by correcting the file name since i forgot the …
EngyNasr Mar 20, 2023
77e2f8c
applying all comments
EngyNasr May 31, 2023
f8bcf61
adding workflows for the collection version
EngyNasr May 31, 2023
121bac4
applying wolfgang comments, still removing the decompress tool is mis…
EngyNasr Jun 20, 2023
4461da3
updating the preprocessing workflow replacing the decompressing step …
EngyNasr Jun 28, 2023
2faea1b
renaming all inputs and outputs to include no spaces and used undersc…
EngyNasr Aug 3, 2023
f3625d8
changing the tested files to be the last file in the collection, to s…
EngyNasr Aug 3, 2023
f7f9c7b
removing the word collection from the folder names, since all workflo…
EngyNasr Aug 3, 2023
486c419
testing another file for SNP workflow to check the testing error
EngyNasr Aug 3, 2023
b3575ec
updating the SNP workflow to solve the testing issue
EngyNasr Aug 4, 2023
033faa0
Second attempt to solve SNP test error
EngyNasr Aug 4, 2023
b4a671c
attempt 1 for solving the taxonomy profiling test failure
EngyNasr Aug 5, 2023
25ee2b8
solving taxonomy profiling testing error
EngyNasr Aug 5, 2023
098f5e0
updating the general readme file to include the training material of …
EngyNasr Aug 7, 2023
54b83e2
using MinusB standard Kraken2 database for taxonomy profiling, trying…
EngyNasr Aug 7, 2023
45847d0
correcting a file name
EngyNasr Aug 7, 2023
0fe88d8
correcing a typo
EngyNasr Aug 7, 2023
8a7e9ae
reducing test datasets samples size, by keeping only reads that are u…
EngyNasr Aug 10, 2023
4551e8c
updating the preprocessing workflow to remove hosts sequences in a mo…
EngyNasr Aug 11, 2023
5f5725f
updating workflows attempt 1 to solve current errors
EngyNasr Aug 11, 2023
0d0b40c
removing filter failed datasets tool, attempt to solve the testing fa…
EngyNasr Aug 11, 2023
db98a38
removing attribute from the genebased test
EngyNasr Aug 11, 2023
fe46a22
trial to solve gene based workflow github test failure
EngyNasr Aug 12, 2023
ac95cf8
attempt 2 to solve the tests failer
EngyNasr Aug 12, 2023
7dfb8d2
updating some workflows
EngyNasr Aug 14, 2023
3d9fab7
correcting a preprocessing workflow test output file
EngyNasr Aug 14, 2023
84855cd
trying to solve genebased test failure
EngyNasr Aug 14, 2023
384588f
another attempt to solve the testing error which states in the log as…
EngyNasr Aug 14, 2023
9624aee
another trial
EngyNasr Aug 14, 2023
7feccfc
solving the taxonomy profiling testing failure due to the standardPF …
EngyNasr Aug 15, 2023
1c949e0
updating workflows based on our latest paper update April 2024
EngyNasr Apr 22, 2024
4267b04
updating workflows to include the tools latest version in Galaxy eu
EngyNasr Apr 23, 2024
79f8553
correcting a test file
EngyNasr Apr 23, 2024
a011293
updating all readme files for workflows
EngyNasr Apr 23, 2024
dbaaa06
adding workflow comments, arranging the main folder and workflows nam…
EngyNasr Apr 25, 2024
f7b7f8a
editing readme and workflows namings and tags based on Berenice's com…
EngyNasr May 8, 2024
c1e6034
solving linting problem
EngyNasr May 9, 2024
7f627a6
updating few typos
EngyNasr May 20, 2024
9aba574
removing a test file which is no longer produced by the updated workf…
EngyNasr May 20, 2024
9d7a9cc
i missed to change the test file correctly the last push, here we go …
EngyNasr May 20, 2024
cc4aa97
updated workflow reports, and adding the total number of reads before…
EngyNasr May 24, 2024
4d0a969
solving the error of MultiQC including the host reads removal details…
EngyNasr May 26, 2024
13d3986
adding a step in the 5th workflow to remove all failed datasets from …
EngyNasr May 26, 2024
308402b
add more filteration steps of removing failed and empty datasets in c…
EngyNasr May 26, 2024
e6aaa69
leaving the optional input to minimap2 selectng the samples profile t…
EngyNasr May 28, 2024
1191598
correcting a comment typo in the preprocessing workflow
EngyNasr May 29, 2024
77ad03d
adding a 5in1 workflow named PathoGFAIR that groups all other 5 workf…
EngyNasr May 31, 2024
11dcc69
removing the 5 in 1 workflow and adding tags to main workflow outputs…
EngyNasr Jun 2, 2024
b6be8a5
correcting typos in readmes
EngyNasr Jun 2, 2024
97d22f7
removing hashtags from tags to make them normal ones not promoted ones
EngyNasr Jun 5, 2024
1c90f5d
Apply suggestions from code review
EngyNasr Jun 19, 2024
f099304
applying all comments from Marius, changing all Readme's accordingly,…
EngyNasr Jun 19, 2024
2b5eced
updating readmes to explain more what to trying workflow mean
EngyNasr Jun 19, 2024
87661e0
adding Zenodo links to all fastq or fasta files
EngyNasr Jun 20, 2024
46ecbc7
Merge branch 'main' of https://github.com/galaxyproject/iwc into micr…
EngyNasr Jun 20, 2024
ecb37b9
updating Zenodo with the correct test file names
EngyNasr Jun 20, 2024
03fd08f
updating a test file name in the gene based test
EngyNasr Jun 20, 2024
05869ee
correcting another test data names
EngyNasr Jun 20, 2024
22505dc
updating resulted test files with the updated file names
EngyNasr Jun 20, 2024
016c352
correcting from path to location for the zenodo tags and adding the f…
EngyNasr Jun 20, 2024
7802d04
nothing changed I am just trying again
EngyNasr Jun 21, 2024
6ae2ded
changing file type for fastq files from fastq to fastqsanger
EngyNasr Jun 21, 2024
d5681ec
using the same fastq test file in Allele based, gene based and taxono…
EngyNasr Jun 21, 2024
00ce14b
updating Preprocessing workflow, to take the reference genome of the …
EngyNasr Jun 21, 2024
b0b0148
removing bar chart tool, to add the reports visualisations later
EngyNasr Jun 24, 2024
cc82985
replacing the testing reference genome since ce6 is giving an error
EngyNasr Jun 24, 2024
25664e2
Thanks to Björn he solved the module issue, I am trying now, hope it …
EngyNasr Jun 24, 2024
7eb0378
2 docker containers have been fixed, thanks to Björn
EngyNasr Jun 25, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions workflows/microbiome/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Microbiome Workflows

In this directory, you will find a collection of workflows designed for microbiome data analysis, pathogen detection, and tracking. These workflows are ready to use and can be adapted for various sequencing techniques using Galaxy's customizable and automatable API.

## Avaiable Workflows

- **Nanopore Preprocessing**

- **Taxonomy Profiling and Visualisation with Krona**

- **Gene-based Pathogen Identification**

- **Allele-based Pathogen Identification**

- **Pathogen Detection: PathoGFAIR Samples Aggregation and Visualisation**

## Getting Started

To learn more about these workflows and to try them with real datasets, please visit our Microbiome tutorials on the Galaxy Training Network (GTN):

[Microbiome Tutorials on GTN](https://training.galaxyproject.org/training-material/topics/microbiome/)


## Dedicated Training Material

The workflows for **Nanopore Preprocessing**, **Taxonomy Profiling and Visualization with Krona**, **Gene-based Pathogen Identification**, **Allele-based Pathogen Identification**, and **Pathogen Detection: PathoGFAIR Samples Aggregation and Visualization** can all be tried out in a dedicated training material on GTN for foodborne pathogen detection and tracking:

[GTN Tutorial for Foodborne Pathogen Detection and Tracking](https://training.galaxyproject.org/training-material/topics/microbiome/tutorials/pathogen-detection-from-nanopore-foodborne-data/tutorial.html)
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
version: 1.2
workflows:
- name: main
subclass: Galaxy
publish: true
primaryDescriptorPath: /Allele-based-Pathogen-Identification.ga
testParameterFiles:
- /Allele-based-Pathogen-Identification-tests.yml
authors:
- name: Engy Nasr
orcid: 0000-0001-9047-4215
url: https://orcid.org/0000-0001-9047-4215
- name: "Bérénice Batut"
orcid: 0000-0001-9852-1987
url: https://orcid.org/0000-0001-9852-1987
- name: Paul Zierep
orcid: 0000-0003-2982-388X
url: https://orcid.org/0000-0003-2982-388X
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
- doc: Test outline for Allele-based-Pathogen-Identification
job:
reference_genome_of_tested_strain:
class: File
location: https://zenodo.org/record/12190648/files/reference_genome_of_tested_strain.fasta.gz
filetype: fasta.gz
collection_of_preprocessed_samples:
class: Collection
collection_type: list
elements:
- class: File
identifier: nanopore_preprocessed_collection_of_all_samples_Spike3bBarcode10
location: https://zenodo.org/record/12190648/files/nanopore_preprocessed_collection_of_all_samples_Spike3bBarcode10.fastq.gz
filetype: fastqsanger.gz
- class: File
identifier: nanopore_preprocessed_collection_of_all_samples_Spike3bBarcode12
location: https://zenodo.org/record/12190648/files/nanopore_preprocessed_collection_of_all_samples_Spike3bBarcode12.fastq.gz
filetype: fastqsanger.gz
samples_profile: null
outputs:
mapping_mean_depth_per_sample:
path: test-data/mapping_mean_depth_per_sample.tabular
Loading
Loading