The Joy of "Screen"

Because I frequently work remotely as well as run long processes I recently gave up my desktop system at work in exchange for a laptop and a couple of virtual machines. I use the laptop for day-to-day work but offload longer computations or automated processes to my VMs. But before I could fully take advantage of this setup I had to solve a new problem I didn't have to worry about with my desktop, how do I run a process remotely without maintaining an open SSH connection?

I went to our IT department with this question and my favorite sys admin¹ turned me onto the joy of Screen. Since then I've been spreading the word around my branch about this awesome little tool. Screen is a robust little piece of software that allows you to manage your shell session in a variety of ways. Screen can do a lot and it's worth taking the time to read through some tutorials but I'll explain my basic workflow in this post to start a process on a remote host and leave it running after closing the SSH connection.

First you SSH into you remote machine as normal and then type

$ screen -ls

This is going to return a list of all the Screen² sessions you have active on your machine. If this is your first time using screen you should see something like No Sockets found in .... Great, now we're going to start up our first Screen session, I'm going to call this one making_science, so then I type:

$ screen -S making_science

And then you'll get a new command line prompt, you are now "in" your Screen session, or as Screen calls it "attached". If you run screen -ls again you'll now get something like this:

there is a screen on:
        48625.making_science    (Attached)
1 Socket in blah-blah

Now go ahead and launch your long script, I usually run it in the background by appending script with an ampersand (&) like this:

$ make_some_science.py &

And now we finally get to the good part, we're going to detach the session without killing the process, even if you close the SSH connection. You can do this either by typing ctrl + a then :detach, or if you don't have those key bindings, screen -D. Anytime you want to check back in one it just reattach your screen session on the same host with screen -r and everything will be just as you left it, even your command history. When you're finally done with your session you can just kill it with exit and it'll be removed from your list of screens.

Lastly, if like me you're suspicious by nature, you're going to want to check to make sure your process is actualy still running so you don't die a little inside when you come back to work the next morning expecting a pile of fresh data and instead get a stack trace. There are a variety of tools that allow you to do this, the two command line solutions I use are top ³ and ps. Either one will list the processed currently running on your machine. I usually start up a process with screen, detach the session, confirm that the process is still running with top or ps and then close the SSH connection. If I'm feeling extra careful I'll check the log files from another host after closing th SSH to make sure things are still humming along.

And that's it. Go ask your IT department very nicely if they can build you a VM and then unleash your codebase while you go about your life. Horray!