HOWTO: Transfer files using Globus Connect

Globus Connect Logo

Globus Connect is a reliable, high-performance file transfer platform allowing users to transfer large amounts of data seamlessly between systems.  It aims to make transfers a "click-and-forget" process by setting up configuration details in the background and automating fault recovery.  

Globus can be used for both file transfers between OSC and:

  • A computing inisitution with Globus installed (check with your site provider for availability) or
  • A personal computer (known as a personal endpoint)

Users transfering between OSC and another computing insitution with Globus installed do not need to install Globus Connect Personal or add OSC certificate authority files, and can skip to Step 3.

More on how Globus works can be found on the Globus "How It Works" page.

If you are looking to transfer smaller sized files you can utilize OnDemand's file transfer capabilities, or use a SFTP client to connect to sftp.osc.edu.

Setup

To use Globus to transfer from a personal computer, you will need to:

  1. Install the Globus Connect Personal client on your computer
  2. Add the certificate authority files to your installation
  3. Create a MyProxy certificate through Glenn

Those transfering between OSC and another computing insitution can skip to Step 3.

1. Installing and Setting Up Globus Connect Personal

  1. Sign up for a free Globus account
  2. Download the Globus Connect Personal Client 
    • Click "Manage Endpoints" under the "Manage Transfers/Data" menu
    • Click "add Globus Connect" on the top-right of the page
    • Choose a unique name for your endpoint and generate the setup key
    • Download the Globus Connect client for your operating system
  3. Install Globus
    • Windows
      1. Run the Installer
      2. Copy-Paste the setup key to complete the installation
    • Mac​​
      1. Mount your drives
      2. Copy the Globus Client to your application Folder
      3. Start The Globus Client, and enter the provided setup key
    • Linux
      1. Un-tar the .tgz file with the command tar -zxvf
      2. Run globusconnect , found within the unzipped directory
      3. Copy-Paste the setup key when prompted

Changing directories accessible to Globus

By default Globus will only add certain default folders to the list of files and directories accessible by Globus.  To change/add/remove files and directories from this list:

Windows

  1. Start Globus Connect Personal
  2. Go to Tools -> Options
  • Add directories/files using the  "+" button
  • Remove directories/files using the "-" button
  • Revert to the default accessible directories/files using the "Reset to Defaults" button
  • Any changes you make are not made permanent until you press the "Save" button

​​Mac

  1. Start Globus Connect Personal
  2. Go to Preferences -> Access
  • Add directories/files using the  "+" button
  • Remove directories/files using the "-" button
  • Revert to the default accessible directories/files using the "Reset to Defaults" button
  • Any changes you make are not made permanent until you press the "Save" button

2. Adding OSC Certificate Authority Files

This process will add Certificate Authority files to your computer to ensure you are a trusted endpoint.  If you installed Globus on a Windows computer, and also installed it to the default directory, we have provided this as a automated process.  Linux and Mac installations, as well as Windows installations to non-default locations, will need to follow the manual instructions below.

  • Automated Process -- Windows Default Installations ONLY

  1. Download the Certificate Authentication zip archive
  2. Unzip the archive 
  3. Run the copy_auth Windows batch file from the unzipped archive
  • You will be prompted whether to allow Windows Command Prompt to make changes to your computer.  Press Yes to continue.
  • If the process was a success, the command prompt should read:

Copy was successful!

  • If an error occurred, the command prompt will read:

Copy was unsuccessful!

       You will need to follow the manual instructions below.

  • Manual Process -- Linux, Mac +  Windows installations to non-default locations

  1. Download the Certificate Authentication zip archive
  2. Unzip the archive
  3. Find your Globus Connect Personal installation directory
    • Mac - located within /Applications/Globus Connect Personal.app/Contents/MacOS/
    • To open the contents of the Globus Connect Personal.app folder, right click the folder in Finder and select "Show Package Contents"
    • Windows - Typically found within /Program Files
    • Linux - Where you unzipped the tarball 
  4. Open the ca directory within the etc directory found in the installation directory
  5. Copy all the files in the unzipped archive, except copy_auth, to the ca directory located previously

3. Creating a MyProxy Certificate 

These steps will create a short lived certificate on OSC's systems to ensure you are a valid OSC user.  This certificate will be good for 0.5 days (11 hours), after which you will need to repeat this process to transfer to/from OSC again.  Repeating this process will overwrite your old certificate, meaning you will need to change the Credentials for osc#Glenn on the Globus website imediately after.

  1. Login to Glenn using your OSC username and password (ssh to Glenn.osc.edu)
  2. Run the command gcp2osc
  • At the first prompt reading "Enter MyProxy pass phrase:" enter your OSC password
  • A second prompt reading "Enter MyProxy pass phrase:" should appear.  You need to enter a new password/passphrase to use to connect to OSC from Globus.  This should be different from your OSC password

Usage

  1. Login to Globus and navigate to the "start transfer" page under the "Manage Transfers" menu
  2. Enter your endpoint in one of the boxes
    • If transferring to a computer with Globus Connect Personal installed, this will be the unique name chosen during installation
  3. Enter osc#glenn as the other endpoint
    • If this is your first time connecting to Glenn through Globus, or if your credentials have expired, you will be prompted to Activate the Endpoint
    • Enter your OSC username
    • Enter the passphrase created during the "Creating a MyProxy Certifcate"
  4. You can now transfer files and directories both ways by selecting them and pressing the arrow indicating which way you'd like to transfer
Once a transfer has begun, you do not need to keep the Globus webpage up, but you will need to make sure the Globus Connect Personal Client is running on your computer until it has completed.  If the transfer is interrupted for any reason, Globus will attempt to re-initiate the transfer automatically.
Supercomputer: