Remote access peace of mind

If you don't already know, each time you open up the terminal, it's considered a "session," and you can have multiple terminal windows open, therefore having multiple sessions. Any commands you run through an open terminal window are considered to be running in that particular session, which is how you can have several different windows doing different things without interacting with each other.

but

But, there is also a built-in layer of protection that if you close a terminal window (an active session), then it immediately ends any processes that were run through that window/session. So, if you were using wget to download a big file and accidentally close the window, then the download gets terminated. yikes. screen is a method (of several) to easily manipulate sessions so closing a window doesn't kill your job/process.

this is especially important for remote (ssh) access

When you remote access something, you are accessing it with a terminal session, and if your connection drops, it disconnects the session, thereby terminating any ongoing jobs/processes that were running in that session. That means any analyses you were running will get prematurely terminated if your computer turns wifi/ethernet off when it goes to sleep, or you may need to leave the campus to do something. So there has to be a way to unchain yourself of absolutely having your session open for long jobs to run.

traditional detachment methods

There are two "classic" methods of taking a job and pushing it into the background so that if you were to close the terminal window or disconnect from ssh that the job would keep running until completion on its own.

bg with disown

bg is a command to put a job in the background, and we often pair it with disown to make it independent of a session. To use this, you have to start the job, then press crtl+z to pause it, then bg, then disown -h. So it looks kind of like:

structure -i infile -o outfile -m mainparams.dat -e extraparams.dat
# then it starts to run and give you some output
# press crtl+z
bg
disown -h

This method usually works, but it's kind of clunky because some software (for whatever reason) gives you only a moment to execute those commands in rapid succession. If you're using dDocent, you will have to use this over nohup because of the yes/no prompting, but screen (below) will work just fine!

nohup

The command nohup is a preferred method to bg. It means "no hangup", and what it does is write any output text from a job into a file called nohup.out (like a log file!) in the active directory. When paired with an &, it also makes the job independent of the session, so you can close your window at will. The syntax is nohup <your full command with arguments> & Just make sure you check the nohup.out file to make sure there weren't any errors in initiating your run, because they will not appear in your terminal window.

# example
nohup structure -i infile -o outfile -m mainparams.dat -e extraparams.dat &

if you are using zsh instead of sh or bash as your shell, then you must end it with &!

# example
nohup structure -i infile -o outfile -m mainparams.dat -e extraparams.dat &!

what is screen?

screen is a very clever and simple command that lets you detach/attach terminal sessions at will. Not all distros have it by default so install it from the available package manager. A great guide can be found here. When you log into an ssh session, run screen

screen

Yeah, it's as easy as that. If you're using it for the first time it will give you a welcome screen, just press return and then it'll appear like a normal terminal again. What you're seeing is kind of like a virtual shell session (a "screen"), but that's not super important. Just run a job as normal

# example
wget http://www.files.com/supergiganticpacbiodatafile.fastq.gz

then press ctrl+A followed by D. Now you've detatched that virtual session and you can close your connection, open up new ones, essentially do whatever you want, and it will just be happily running in the background. Please see the guide about naming screens.

reattach a screen

screen -r <screen name>
# if you just have one screen, you won't need to include a name

list all available screens

screen -ls