diff --git a/_quarto.yml b/_quarto.yml index e78619b..3c3fd68 100644 --- a/_quarto.yml +++ b/_quarto.yml @@ -86,36 +86,28 @@ website: - docs/lesson02-using-the-command-line/01-navigating-file-directories.qmd - docs/lesson02-using-the-command-line/02-working-with-file.qmd - docs/lesson02-using-the-command-line/03-redirection.qmd - - section: "QC & Assembly" - href: docs/lesson03-qc-assembly/index.qmd + - section: "QC & RNA pre-processing" + href: docs/lesson03-qc-pre-processing/index.qmd contents: - - docs/lesson03-qc-assembly/01-introduction-meta.qmd - - docs/lesson03-qc-assembly/02-QC-quality-raw-reads.qmd - - docs/lesson03-qc-assembly/03-assembly.qmd - - section: "Polishing" - href: docs/lesson04-polishing/index.qmd + - docs/lesson03-qc-pre-processing/01-introduction-meta.qmd + - docs/lesson03-qc-pre-processing/02-QC-raw-reads.qmd + - docs/lesson03-qc-pre-processing/03-rrna-filtering.qmd + - section: "Taxonomic Annotation" + href: docs/lesson04-taxonomic-annotation/index.qmd contents: - - docs/lesson04-polishing/01-polishing-assembly.qmd - - docs/lesson04-polishing/02-QC-polished-assembly.qmd - - section: "Binning & Functional Annotation" - href: docs/lesson05-binning-functional-annotation/index.qmd + - docs/lesson04-taxonomic-annotation/01-community-structure.qmd + - docs/lesson04-taxonomic-annotation/02-visualising-structure.qmd + - section: "Functional Annotation" + href: docs/lesson05-functional-annotation/index.qmd contents: - - docs/lesson05-binning-functional-annotation/01-binning.qmd - - docs/lesson05-binning-functional-annotation/02-binning_quality.qmd - - docs/lesson05-binning-functional-annotation/03-Functional-annotation.qmd - - section: "Taxonomic Annotations" - href: docs/lesson06-taxonomic-annotations/index.qmd + - docs/lesson05-functional-annotation/01-functional-info.qmd + - docs/lesson05-functional-annotation/02-normalising-and-identifying.qmd + - section: "Combining Annotations" + href: docs/lesson06-combining-annotations/index.qmd contents: - - docs/lesson06-taxonomic-annotations/01-taxonomic.qmd - - docs/lesson06-taxonomic-annotations/02-Diversity-tackled-with-R.qmd - - docs/lesson06-taxonomic-annotations/03-hands_on-diversity.qmd - # - section: "Automating Analyses with Bash Scripts" - # href: docs/lesson07-automation-bash-scripts/index.qmd - # contents: - # - docs/lesson07-automation-bash-scripts/01-scripting-intro.qmd - # - docs/lesson07-automation-bash-scripts/02-base-automation-script.qmd - # - docs/lesson07-automation-bash-scripts/03-adding-logging-and-control.qmd - # - docs/lesson07-automation-bash-scripts/04-creating-module-based-version.qmd + - docs/lesson06-combining-annotations/01-combining-annotations.qmd + - docs/lesson06-combining-annotations/02-Diversity-tackled-with-R.qmd + - docs/lesson06-combining-annotations/03-hands_on-diversity.qmd - section: "Extras" contents: - docs/miscellanea/extras/data.qmd diff --git a/docs/lesson03-qc-assembly/01-introduction-meta.qmd b/docs/lesson03-qc-pre-processing/01-introduction-meta.qmd similarity index 100% rename from docs/lesson03-qc-assembly/01-introduction-meta.qmd rename to docs/lesson03-qc-pre-processing/01-introduction-meta.qmd diff --git a/docs/lesson03-qc-assembly/02-QC-quality-raw-reads.qmd b/docs/lesson03-qc-pre-processing/02-QC-raw-reads.qmd similarity index 100% rename from docs/lesson03-qc-assembly/02-QC-quality-raw-reads.qmd rename to docs/lesson03-qc-pre-processing/02-QC-raw-reads.qmd diff --git a/docs/lesson03-qc-assembly/03-assembly.qmd b/docs/lesson03-qc-pre-processing/03-rrna-filtering.qmd similarity index 100% rename from docs/lesson03-qc-assembly/03-assembly.qmd rename to docs/lesson03-qc-pre-processing/03-rrna-filtering.qmd diff --git a/docs/lesson03-qc-assembly/index.qmd b/docs/lesson03-qc-pre-processing/index.qmd similarity index 100% rename from docs/lesson03-qc-assembly/index.qmd rename to docs/lesson03-qc-pre-processing/index.qmd diff --git a/docs/lesson04-polishing/01-polishing-assembly.qmd b/docs/lesson04-taxonomic-annotation/01-community-structure.qmd similarity index 100% rename from docs/lesson04-polishing/01-polishing-assembly.qmd rename to docs/lesson04-taxonomic-annotation/01-community-structure.qmd diff --git a/docs/lesson04-polishing/02-QC-polished-assembly.qmd b/docs/lesson04-taxonomic-annotation/02-visualising-structure.qmd similarity index 100% rename from docs/lesson04-polishing/02-QC-polished-assembly.qmd rename to docs/lesson04-taxonomic-annotation/02-visualising-structure.qmd diff --git a/docs/lesson04-polishing/index.qmd b/docs/lesson04-taxonomic-annotation/index.qmd similarity index 100% rename from docs/lesson04-polishing/index.qmd rename to docs/lesson04-taxonomic-annotation/index.qmd diff --git a/docs/lesson05-binning-functional-annotation/01-binning.qmd b/docs/lesson05-functional-annotation/01-functional-info.qmd similarity index 100% rename from docs/lesson05-binning-functional-annotation/01-binning.qmd rename to docs/lesson05-functional-annotation/01-functional-info.qmd diff --git a/docs/lesson05-binning-functional-annotation/02-binning_quality.qmd b/docs/lesson05-functional-annotation/02-normalising-and-identifying.qmd similarity index 100% rename from docs/lesson05-binning-functional-annotation/02-binning_quality.qmd rename to docs/lesson05-functional-annotation/02-normalising-and-identifying.qmd diff --git a/docs/lesson05-binning-functional-annotation/03-Functional-annotation.qmd b/docs/lesson05-functional-annotation/03-Functional-annotation.qmd similarity index 100% rename from docs/lesson05-binning-functional-annotation/03-Functional-annotation.qmd rename to docs/lesson05-functional-annotation/03-Functional-annotation.qmd diff --git a/docs/lesson05-binning-functional-annotation/index.qmd b/docs/lesson05-functional-annotation/index.qmd similarity index 100% rename from docs/lesson05-binning-functional-annotation/index.qmd rename to docs/lesson05-functional-annotation/index.qmd diff --git a/docs/lesson06-taxonomic-annotations/01-taxonomic.qmd b/docs/lesson06-combining-annotations/01-combining-annotations.qmd similarity index 100% rename from docs/lesson06-taxonomic-annotations/01-taxonomic.qmd rename to docs/lesson06-combining-annotations/01-combining-annotations.qmd diff --git a/docs/lesson06-taxonomic-annotations/02-Diversity-tackled-with-R.qmd b/docs/lesson06-combining-annotations/02-Diversity-tackled-with-R.qmd similarity index 100% rename from docs/lesson06-taxonomic-annotations/02-Diversity-tackled-with-R.qmd rename to docs/lesson06-combining-annotations/02-Diversity-tackled-with-R.qmd diff --git a/docs/lesson06-taxonomic-annotations/03-hands_on-diversity.qmd b/docs/lesson06-combining-annotations/03-hands_on-diversity.qmd similarity index 100% rename from docs/lesson06-taxonomic-annotations/03-hands_on-diversity.qmd rename to docs/lesson06-combining-annotations/03-hands_on-diversity.qmd diff --git a/docs/lesson06-taxonomic-annotations/index.qmd b/docs/lesson06-combining-annotations/index.qmd similarity index 100% rename from docs/lesson06-taxonomic-annotations/index.qmd rename to docs/lesson06-combining-annotations/index.qmd diff --git a/docs/lesson07-automation-bash-scripts/01-scripting-intro.qmd b/docs/lesson07-automation-bash-scripts/01-scripting-intro.qmd deleted file mode 100644 index a048781..0000000 --- a/docs/lesson07-automation-bash-scripts/01-scripting-intro.qmd +++ /dev/null @@ -1,7 +0,0 @@ ---- -title: "Scripting Basics" -page-navigation: false ---- -# Overview - -in construction \ No newline at end of file diff --git a/docs/lesson07-automation-bash-scripts/02-base-automation-script.qmd b/docs/lesson07-automation-bash-scripts/02-base-automation-script.qmd deleted file mode 100644 index cb46cd3..0000000 --- a/docs/lesson07-automation-bash-scripts/02-base-automation-script.qmd +++ /dev/null @@ -1,7 +0,0 @@ ---- -title: "Base Environmental Metagenomics Script" -page-navigation: false ---- -# Overview - -in construction diff --git a/docs/lesson07-automation-bash-scripts/03-adding-logging-and-control.qmd b/docs/lesson07-automation-bash-scripts/03-adding-logging-and-control.qmd deleted file mode 100644 index c5cef89..0000000 --- a/docs/lesson07-automation-bash-scripts/03-adding-logging-and-control.qmd +++ /dev/null @@ -1,7 +0,0 @@ ---- -title: "Adding Logging and Control to the Base Script" -page-navigation: false ---- -# Overview - -in construction \ No newline at end of file diff --git a/docs/lesson07-automation-bash-scripts/04-creating-module-based-version.qmd b/docs/lesson07-automation-bash-scripts/04-creating-module-based-version.qmd deleted file mode 100644 index f743b66..0000000 --- a/docs/lesson07-automation-bash-scripts/04-creating-module-based-version.qmd +++ /dev/null @@ -1,7 +0,0 @@ ---- -title: "Creating a Module-based Script" -page-navigation: false ---- -# Overview - -in construction \ No newline at end of file diff --git a/docs/lesson07-automation-bash-scripts/index.qmd b/docs/lesson07-automation-bash-scripts/index.qmd deleted file mode 100644 index 23da48f..0000000 --- a/docs/lesson07-automation-bash-scripts/index.qmd +++ /dev/null @@ -1,26 +0,0 @@ ---- -title: "Automating Analyses with Bash Scripts" -subtitle: "[Metagenomics Environmental Application](/index.qmd)" -# title-block-banner: true ---- -## Introduction - -This lesson is an introduction to computer task automation using Bash scripts. Using the data management and data analysis tasks that you manually ran in the previous lessons as a running example, this lesson shows how to create Bash scripts that will run those tasks automatically one after the other. Thus, you will only need to run a single script to get done the tasks your analysis require. - -A script is a file that contains all the tasks that you want to be performed, that is: all the shell **commands** (e.g.: `ls`, `mkdir`, `rm`, etc.) and **programs** (e.g., `fastqc`, `flye`, `pilon`, etc.) that need to be run to get your analysis done. - -When you run a script, the shell will open the script file and run each of the **commands** and **programs** in the script one after the other, either in the foreground or in the background if you specify the backgroud operator `&` after the script name. - -Scripts consisting of various tasks are also referred to as **workflows**. Bash workflow scripting is widely used to automate software installs, software updates, data backups, among many others tasks. - -## Overview -The lesson consists of four episodes: - -- **Scripting Basics** --- covers how to create a script, a few simple script examples, how to run scripts, and how and when to make a script runnable from any location within the file system hierarchy. - -- **Base Environmental Metagenomics Script** --- shows a workflow script that runs all of the tasks that you ran in the previous lessons **except** the `R` tasks in the last lesson and the tasks (commands) that you ran in your local machine, for example the `scp` commands you ran in your local machine to copy files from your AWS instance to your local machine. This workflow script is called `cswf01_baseEnvmntlMetaGenomics.sh`. - -- **Adding Logging and Control to the Base Script** --- shows how to add both logging information on the progress of a workflow script and execution control on the outcome of the script tasks. Taking the base workflow script (`cswf01_base..sh`) as a starting point, this episode shows how to add shell commands to both write onto a log file the time at which each task starts and finishes and to check whether each task runs successfully, aborting the script if a task is unsuccessful. It is **best practise** to abort a workflow script as soon as a task is unsuccessful as subsequent tasks will have either no data or wrong data and there is no point in waiting for bad results. The resulting workflow script is called `cswf2_logAndCtrlEnvmntlMetaGenomics.sh`. - -- **Creating a Module-based Script** --- covers how to create a modular version of the previous script (`cswf02_log..sh`). A modular version is split into various scripts, each script being responsible for performing a specific task. The advantages of modular scripting (and software development at large) include: module reuse and hence faster development of new scripts, and cleaner and simpler coding. The episode shows how to create (1) a script out of each of the program-based tasks in the previous script, and (2) a script that runs the program-based task scripts to accomplish the full functionality of the previous script. Program-based tasks correspond to those that run analysis tools, `fastqc`, `flye`, `pilon`, etc. The (program-based) module scripts will enable you to create new workflow scripts by simply invoking the module scripts that your workflow needs. -