Error with kernel swap limit/ cgroup while executing NFCORE_CUTANDRUN:CUTANDRUN:SAMTOOLS_CUSTOMVIEW

[Prathyusha-konda]

Description of the bug

Hi, thank you for developing this pipeline. It has exactly all the steps I am interested in. I am receiving the following error related to swap limit/ cgroup and the execution halts within the first couple of steps. (We are getting the same error for RNASeq pipeline too FYI). I am running the pipeline on our university HPC and they informed me that they cannot update docker or kernel. Do you have any suggestions for how to work around these errors and run the pipeline without swap limits? I am also attaching the log if that is of any help.

Here is my command:
nextflow run nf-core/cutandrun -profile docker --input CUT_AND_RUN_SAMPLES_03_27_2024.csv --peakcaller macs2 --genome hg38 --blacklist hg38-blacklist.v2.bed --outdir nextflow_output

Thank you very much!

Command used and terminal output

Mar-30 23:44:06.626 [main] INFO  nextflow.cli.CmdRun - Launching `
https://github.com/nf-core/cutandrun`
[soggy_ramanujan] DSL2 - revision: 6e1125d4fe [master]
Mar-30 23:44:06.628 [main] WARN  nextflow.plugin.PluginsFacade - Nextflow self-contained distribution allows only core plugins -- User config plugins will be ignored: nf-validation@1.1.3


Mar-30 23:46:34.945 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for
  task: name=NFCORE_CUTANDRUN:CUTANDRUN:SAMTOOLS_CUSTOMVIEW (NT4M_V5_R1); work-dir=/home/pk576/Alessandro_cut_and_run/Mar2024/work/47/8f27135328f2bf8acacff6d15bb04a
  error [nextflow.exception.ProcessFailedException]: Process `NFCORE_CUTANDRUN:CUTANDRUN:SAMTOOLS_CUSTOMVIEW (NT4M_V5_R1)` terminated with an error exit status (1)
Mar-30 23:46:35.088 [Task monitor] ERROR nextflow.processor.TaskProcessor - Error executing process > 'NFCORE_CUTANDRUN:CUTANDRUN:SAMTOOLS_CUSTOMVIEW (NT4M_V5_R1)'
 
Caused by:
  Process `NFCORE_CUTANDRUN:CUTANDRUN:SAMTOOLS_CUSTOMVIEW (NT4M_V5_R1)` terminated with an error exit status (1)
 
Command executed:
 
  samtools view -F 0x04 -@ 2 NT4M_V5_R1.target.markdup.sorted.bam | awk -F'\t' 'function abs(x){return ((x < 0.0) ? -x : x)} {print abs($9)}' | sort -T '.' | uniq -c | awk -v OFS="\t" '{print $2, $1/2}' > NT4M_V5_R1.frags.len.txt
 
  cat <<-END_VERSIONS > versions.yml
  "NFCORE_CUTANDRUN:CUTANDRUN:SAMTOOLS_CUSTOMVIEW":
      samtools: $(echo $(samtools --version 2>&1) | sed 's/^.*samtools //; s/Using.*$//')
  END_VERSIONS
 
Command exit status:
  1
 
Command output:
  (empty)
 
Command error:
  WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
  [E::hts_tpool_init] Couldn't start thread pool worker : Operation not permitted
  Error creating thread pool

Relevant files

nextflow.cutandrun.log

System information

Ubuntu16.04
kernel:
Linux argos8 4.4.0-173-generic #203-Ubuntu SMP Wed Jan 15 02:55:01 UTC 2020 x86_64 x86_64 x86_64 GNU/Linu

[chris-cheshire]

Hi, sorry for the delay, I changed jobs and I am only just catching up. Firstly, these questions are best posted on nf-core slack channels so everyone can see them and comment. This looks like your host system has run out of memory or docker has.

How much memory is available to you run this pipeline?

[Prathyusha-konda]

Hi, sorry for the delay, I changed jobs and I am only just catching up. Firstly, these questions are best posted on nf-core slack channels so everyone can see them and comment. This looks like your host system has run out of memory or docker has.

How much memory is available to you run this pipeline?

Hi Chris,

Thank you for your response. I am using a HPC so memory availability should not be a problem I think. Here is the availability on one of the computational nodes.

X total used free shared buff/cache available
Mem 394842196 100747636 36627596 83216 257466964 290780484
Swap 8788988 3113932 5675056

Is there is a specific memory requirement for the docker? ? If yes, I could specify in "docker run --memory --memory-swap"? Please let me know what you think. I can also ask on slack (I was told that seqero might be a more appropriate place so may be there too). Thanks a lot!