CAT on AWS

[larryns]

Hi,

I'm running cat on AWS nodes. I'm having several problems:

First, when I tried to launch toil, it timed out connecting to the flatcar AMI server. So I set the flatcar AMI to: ami-08870dfc70c8d6fe4.

Second, it looks like docker won't mount any of the directories I created. I ran docker outside of cat, and still had this problem. For example:
docker run -v /catjobs:/catjobs -u 0:0 -i -t --rm quay.io/ucsc_cgl/cat:latest /bin/bash

results in an empty /catjobs. The only way I could work around this, is to put my catjobs directory in /var/tmp. It seems to mount those directories, but not the ones I create. I checked permissions and am running as root, so didn't see a problem there.

Third, the biggest problem I have is I'm getting the following error.
INFO: Informed scheduler that task Chaining_False_True_True_1fdbe7409d has status FAILED

GenerateHints also failed, but I don't see any other useful messages or information. I've also looked in the luigi_logs but I don't see what went wrong.

I've run:

luigid --background --logdir luigi_logs```

and my CAT command is:

luigi
--module cat RunCat
--hal="mMolNig1.hal"
--ref-genome="hg38"
--config="cat.config"
--work-dir="cat-mMolNig1"
--out-dir="cat-mMolNig1-out"
--workers=3
--maxCores=8
--augustus
--augustus-pb
--augustus-cgp
--assembly-hub
--batchSystem mesos
--zone "us-east-1a"
--nodeTypes "m6a.8xlarge,c6a.8xlarge,c6a.16xlarge,r6a.8xlarge"
--provisioner aws
--maxNodes 20

Thanks for any help.
Larry.