Common Errors

SSH Disconnection While Pipeline is Running

It is recommended that you run all snakemake commends in a tmux session, so they will continue running even if you get disconnected.

Some basic commands to get started are:

• tmux new -s snakemake starts a new session
• ./run_pipeline.sh ... run the snakemake pipeline here
• crtl+b d detaches from a current session
• tmux ls
• tmux attach -t snakemake reattaches to a session
• exit kills the tmux session if you are inside it
• tmux kill-session -t snakemake kills the tmux session if outside of it

Out of memory error:

If slurm runs out of memory, it might give an error similar to these:

slurmstepd: error: Detected 2 oom_kill events in StepId=12475604.batch. Some of the step tasks have been OOM Killed.

Error in unserialize(node$con) : MultisessionFuture (<none>) failed to receive message results from cluster RichSOCKnode

If this is the case, you can edit the corresponding rule in either

• snakemake/<specific_pipeline>.smk
• snakemake/rules/<specific_rule>.smk

You will see a resources block with mem. You can increase the values in that if you need to.

Locked directory

If snakemake is cancelled halfway through execution, the directory can be left in a locked state:

LockException:
Error: Directory cannot be locked. Please make sure that no other Snakemake process is trying to create the same files in the following directory:

To unlock, please add --unlock to the snakemake or ./run_pipeline.sh command, this will unlock, and you can run your previous command as normal.