Skip to content

Latest commit

 

History

History
114 lines (78 loc) · 4.25 KB

benchmarks.md

File metadata and controls

114 lines (78 loc) · 4.25 KB
Raw

Benchmarks:

Benchmarks were run 5x each with 1 warmup run - using hyperfine.

They were run on my own PC running Ubuntu 23.04:

  • i7 12700k - 12 cores
  • 32GB memory
  • M.2 SSD drive

Hyperfine command:

hyperfine --warmup 1 -r 5 --export-json --export-markdown

Unpaired

Single tests were based on the Zymo mock community Zymo-GridION-EVEN-BB-SN dataset.

Benchmark 1:

ONT long read sequencing, input fastq format, outputting a non-compressed fastq file. Extracting all reads classified as Bacillus spizizenii.

Inputs:

File type Platform Total reads Reads to extract Kraken output size Output
.fastq ONT 3,491,078 490,984 885MB .fastq

Commands run:

Tool Command
KrakenTools extract_kraken_reads.py -s Zymo-GridION-EVEN-BB-SN.fq -k out.kraken -o krakentools.fq -t 96241 --fastq-output
Kractor kractor -i Zymo-GridION-EVEN-BB-SN.fq -k out.kraken -o kractor.fq -t 96241

Results:

Tool Mean [s] Min [s] Max [s] Relative
krakenTools 254.881 ± 9.482 242.553 263.158 1.00
Kractor 58.253 ± 5.651 48.358 62.222 4.38 ± 0.45

Benchmark 2:

ONT long read sequencing, input fastq.gz format, outputting a non-compressed fastq file. Extracting all reads classified as Bacillus spizizenii.

Inputs:

File type Platform Total reads Reads to extract Kraken output size Output
.fastq.gz ONT 3,491,078 490,984 885MB .fastq

Commands run:

Tool Command
KrakenTools extract_kraken_reads.py -s Zymo-GridION-EVEN-BB-SN.fq.gz -k out.kraken -o krakentools.fq -t 96241 --fastq-output
Kractor kractor -i Zymo-GridION-EVEN-BB-SN.fq.gz -k out.kraken -o kractor.fq -t 96241

Results:

Command Mean [s] Min [s] Max [s] Relative
krakenTools 376.592 ± 3.905 373.343 383.315 1.00
Kractor 100.044 ± 3.599 98.044 106.449 3.76 ± 0.14

Paired

Paired end tests were based on SRA accession: SRR19995508

Benchmark 3:

Illumina paired end sequencing, input fastq format, outputting a non-compressed fastq file.

Inputs:

File type Platform Total reads Reads to extract Kraken output size Output
.fastq Illumina paired 53,526,611 1,646,117 9.3GB .fastq

Commands run:

Tool Command
KrakenTools extract_kraken_reads.py -s SRR19995508_R1.fastq -s2 SRR19995508_R2.fastq -o R1_tools.fq -o2 R2_tools.fq -k out.kraken -t 590 --fastq-output
Kractor kractor -i SRR19995508_R1.fastq -i SRR19995508_R2.fastq -k out.kraken -t 590 -o R1_kractor.fq -o R2_kractor.fq

Results:

Command Mean [s] Min [s] Max [s] Relative
krakenTools 898.306 ± 14.203 884.229 920.653 1.00
Kractor 94.198 ± 2.317 90.852 96.474 9.54 ± 0.28

Benchmark 4:

Illumina paired end sequencing, input fastq.gz format, outputting a non-compressed fastq file.

Inputs:

File type Platform Total reads Reads to extract Kraken output size Output
.fastq.gz Illumina paired 53,526,611 1,646,117 9.3GB .fastq

Commands run:

Tool Command
KrakenTools extract_kraken_reads.py -s SRR19995508_R1.fastq.gz -s2 SRR19995508_R2.fastq.gz -o R1_tools.fq -o2 R2_tools.fq -k out.kraken -t 2 --fastq-output
Kractor kractor -i SRR19995508_R1.fastq.gz -i SRR19995508_R2.fastq.gz -k out.kraken -t 2 -o R1_kractor.fq -o R2_kractor.fq

Results:

Command Mean [s] Min [s] Max [s] Relative
krakenTools 1033.379 ± 25.238 1005.720 1068.522 1.00
Kractor 49.071 ± 0.179 48.857 49.334 21.06 ± 0.52