How to re-run the job?

[Windyxia11]

My task has been running for half a month and suddenly stopped, because the genome size exceeds the processing limit. How can I solve this problem, in this case, is there any way to re-run the results of the previous run? Because the comparison time is too long, or better advice?

My error information as follows:
[2025-01-10T16:01:11+0800] [MainThread] [W] [toil.fileStores.abstractFileStore] Downloaded file 'files/for-job/kind-PreprocessSequence/instance-oo8xel_w/cleanup/file-d78f0519b8ce493cbd01229ee40a6e2b/197.fa' to path '/tmp/4bba38855f7a530eb9937f99d5f5d5fb/aec0/6109/tmpdh97mzmb/Saccharum_hybrid_cultivar_ZZ1_11_197.tgt'
Traceback (most recent call last):
File "/xtdisk/zhaowm_bigd/kanghl/software/Cactus/cactus-bin-v2.7.0/venv-cactus-v2.7.0/lib/python3.8/site-packages/toil/worker.py", line 403, in workerScript
job._runner(jobGraph=None, jobStore=jobStore, fileStore=fileStore, defer=defer)
File "/xtdisk/zhaowm_bigd/kanghl/software/Cactus/cactus-bin-v2.7.0/venv-cactus-v2.7.0/lib/python3.8/site-packages/cactus/shared/common.py", line 960, in _runner
super(RoundedJob, self)._runner(*args, jobStore=jobStore,
File "/xtdisk/zhaowm_bigd/kanghl/software/Cactus/cactus-bin-v2.7.0/venv-cactus-v2.7.0/lib/python3.8/site-packages/toil/job.py", line 2774, in _runner
returnValues = self._run(jobGraph=None, fileStore=fileStore)
File "/xtdisk/zhaowm_bigd/kanghl/software/Cactus/cactus-bin-v2.7.0/venv-cactus-v2.7.0/lib/python3.8/site-packages/toil/job.py", line 2691, in _run
return self.run(fileStore)
File "/xtdisk/zhaowm_bigd/kanghl/software/Cactus/cactus-bin-v2.7.0/venv-cactus-v2.7.0/lib/python3.8/site-packages/cactus/preprocessor/lastzRepeatMasking/cactus_lastzRepeatMask.py", line 215, in run
alignment = self.alignFastaFragments(fileStore, targetFiles, fragments)
File "/xtdisk/zhaowm_bigd/kanghl/software/Cactus/cactus-bin-v2.7.0/venv-cactus-v2.7.0/lib/python3.8/site-packages/cactus/preprocessor/lastzRepeatMasking/cactus_lastzRepeatMask.py", line 94, in alignFastaFragments
cactus_call(outfile=alignment,
File "/xtdisk/zhaowm_bigd/kanghl/software/Cactus/cactus-bin-v2.7.0/venv-cactus-v2.7.0/lib/python3.8/site-packages/cactus/shared/common.py", line 891, in cactus_call
raise RuntimeError("{}Command {} exited {}: {}".format(sigill_msg, call, process.returncode, out))
RuntimeError: Command ['lastz', 'Saccharum_hybrid_cultivar_ZZ1_11.fa[multiple][nameparse=darkspace]', 'Saccharum_hybrid_cultivar_ZZ1_11_frag[nameparse=darkspace]', '--step=3', '--ambiguous=iupac,100,100', '--ungapped', '--queryhsplimit=keep,nowarn:1500', '--querydepth=keep,nowarn:13', '--format=general:name1,zstart1,end1,name2,zstart2+,end2+', '--markend'] exited 1: stderr=FAILURE: in load_fasta_sequence for Saccharum_hybrid_cultivar_ZZ1_11.fa, sequence length 2,145,801,589+10,000,000 exceeds maximum (2,147,483,637)

    [2025-01-10T16:01:11+0800] [MainThread] [E] [toil.worker] Exiting the worker because of a failed job on host fatltd2

Any help will be greatly appreciated!

[glennhickey]

Cactus can be resumed with the --restart option, but that will only run the same logic again which will fail in the same way.

But you are still in the preprocessing stage after half a month, so I don't think there is much hope for this run. If you start again with the latest version of cactus (the failing job from cactus_lastzRepeatMask.py was deprecated in v2.8.0),, it will only take a few hours to advance past this stage of the pipeline.

To give more flexibility in resuming after future catastrophic errors, you may want to consider the step-by-step pipline

https://github.com/ComparativeGenomicsToolkit/cactus/blob/master/doc/progressive.md#running-step-by-step