Docker Spawner & persistent storage

The following steps summarize how to use the Docker Spawner & deploy persistent storage on a server that has Jubyterhub (TLJH) installed.

1- Create a new storage device on AWS by creating a new volume and attaching it to the instance.

2- Use the command ls /dev on the terminal to check if the new volume was attached successfully to the instance. The new disk named xvdf should appear.

3- install zfs using sudo apt install zfsutils-linux

4- add a user called jupyter using:

sudo useradd -g 9999 jupyter
sudo chown -R 9999:9999 /tank/home/dsci100

4- Edit the file /opt/tljh/config/jupyterhub_config.d/jupyterhub_config.py to include the following lines

def create_fs_hook(spawner):
      username = spawner.user.name
      callysto_user = 'jupyter'
      course = 'dsci100'
      check_call(['/srv/jupyterhub/zfs_homedir.sh', course, username, callysto_user])

c.Spawner.pre_spawn_hook = create_fs_hook
c.DockerSpawner.volumes = {'/tank/home/dsci100/{username}': '/home/jupyter'}

5- create a new file in /srv/jupyterhub/ called zfs_homedir.sh. It should contain the code:

#!/usr/bin/env bash

set -e
set -u
set -o pipefail

ZFSHOME="tank/home"
ZFSOPTS="-o refquota=1G"

sys_user_exists() {
    local username="$1"
    ret=false
    getent passwd ${username} > /dev/null 2>&1 && ret=true
    if ! ${ret}; then
        echo "username \"${username}\" does not correspond to a system user!" >&2
        echo >&2
        exit 1
    fi
}

main() {
    local username

    local zfshome="tank/home"

    # Accept three and only three arguments
    if [[ -z "${1:-}" || -z "${2:-}" || -z "${3:-}" ]]; then
        echo "Usage ${0} <COURSENAME> <USERNAME> <OWNER>" >&2
        echo >&2
        exit 1
    elif [[ ${1} =~ ^[a-z0-9][_-a-z0-9.]*$ || ${1} =~ ^[a-z0-9][-a-z0-9.]*$ ]]; then
        coursename=${1}
        username=${2}
	owner=${3}
    else
        echo "Coursename \"${1}\" must match schema ^[a-z0-9][_-a-z0-9.]*\$" >&2
        echo "Username \"${2}\" must match schema ^[a-z0-9][-a-z0-9.]*\$" >&2
        echo >&2
        exit 1
    fi
    sys_user_exists $owner
    
    local homefs="${zfshome}/${coursename}/${username}"
    local homedir="/$homefs"
    if [[ -d "${homedir}" ]] ; then
        echo "Homedir already exists, not recreating"
    else
        echo "Creating Homedir: ${homedir}"
        zfs create ${ZFSOPTS} ${homefs}
        chown -R ${owner}:${owner} ${homedir}
	chmod g+w ${homedir}
    fi
}

main "${@:-}"

6- do the following edits in the file /etc/systemd/system/jupyterhub.service

PrivateDevices=no
ProtectedKernelModules=no
Type=simple

7- Make sure all Docker containers are stopped and removed when testing for persistent storage... Test this via sudo docker ps -a

Stop docker containers:

sudo docker stop <CONTAINER_ID>

Delete docker containers:

sudo docker rm <CONTAINER_ID>

8- Restart jupyter hub using

systemctl daemon-reload
systemctl restart jupyterhub

9- create a new zfs pool using zfs create /tank/home/<courseName>/<userName> example: zfs create /tank/home/dsci100/test

10- check the zfs pool was created successfully using zpool list.

11- If you want to delete the created zfs pool use sudo zfs destroy tank/home/dsci100/test.