Transferring Data Between Savio and Your UC Berkeley Box Account

This document describes how you can transfer files between Savio, UC Berkeley's high performance computing cluster, and your UC Berkeley bConnected account on the Box collaboration/file storage service, via the FTPS file transfer protocol.

Via the instructions below, you'll perform these file transfers using the lftp utility on Savio, which supports the FTPS protocol.

Perform one-time setup

You'll only need to complete the following steps once:

Set up an "external password" on your UC Berkeley Box account. This which enables your account to be accessed via methods other than single sign-on, such as via the FTPS protocol.

1. Log into your Box account (https://box.berkeley.edu) via CalNet authentication.

2. From the dropdown menu with your name, near the upper right, select Account Settings.

3. Scroll down to the Create External Password section.

4. Click the Edit Password link.

5. Follow the onscreen prompts to create and save your external password.

Open a connection from Savio to Box

On Savio, use lftp to open an encrypted (FTPS) connection to Box:

1. Via SSH, log into the Data Transfer Node on Savio, dtn.brc.berkeley.edu

   E.g. for a command line SSH client (substituting your actual Savio username for my_savio_user_name below):

   ssh my_savio_user_name@dtn.brc.berkeley.edu

2. Launch lftp and connect to Box by entering:

   lftp ftp.box.com

   (You should then see an lftp prompt, similar to lftp my_email@berkeley.edu@ftp.box.com:~> or lftp ftp.box.com:/my/directory_path_on_box> The following commands are all typed at that lftp prompt, after the > symbol.)

3. To ensure that your username and password are encrypted when being sent from Savio to Box - an important security precaution - enter:

   set ssl-allow true

4. To enter your username and password, enter (substituting your actual UC Berkeley email address, used with your Box account, for my_email@berkeley.edu below):

   user my_email@berkeley.edu

5. When prompted for a password, enter your Box external password. (This is the password that you set up via steps earlier in these instructions.)

Transfer files between Savio and Box

On Savio, you'll use lftp commands to transfer files in either direction, between Savio and Box. You'll enter each of the following commands after lftp's > prompt.

Note: the full prompt might look something more like this:

lftp my_email@berkeley.edu@ftp.box.com:~>

Or this:

lftp my_email@berkeley.edu@ftp.box.com:/my/directory_path_on_box>

The basics ... some of the commands you'll use most often

• Show the name of my current local directory (on Savio):

  lpwd

• Show me what's in my current Box directory:

  ls

• Change to a different Box directory:

  cd box_directory_name

  (Hint: if the Box directory name contains space characters or other whitespace, enter any unique first part of that name and press the Tab key. This will fill in the remainder of that name.)

• Transfer a single file from Box to your current directory on Savio:

  get myfile.xyz

• Transfer a single file from Savio to your current directory on Box. (This example assumes the file myotherfile.xyz is in your current directory on Savio):

  put myotherfile.xyz

• Transfer multiple files from Box to your current directory on Savio (examples):

  mget some_filename_pattern_with_wildcards*

  mget some_filename*pattern*with_wildcards

  (You can think of the mget command as "multiple file get".)

• Transfer multiple files from Savio to your current directory on Box. (This example assumes these files are all in your current directory on Savio):

  mput some_filename_pattern_with_wildcards*

  mput some_filename*pattern*with_wildcards

  (You can think of the mput command as "multiple file put".)

Some other potentially useful commands

• Show me what's in my current Savio directory:

  !ls

  (Note: that's a exclamation mark - that is, an ! character - in front of the ls command. Use ! followed by any shell command to execute that command from inside lftp.)

• Change to a different local directory (on Savio):

  lcd savio_directory_name