2022 Storage Service Upgrades

In October 2022, OSC retires the Data Direct Networks (DDN) GRIDScaler system deployed in 2016 and expands the IBM Elastic Storage System (ESS) for both Project and global Scratch services. This expands the total capacity of Project and Scratch storage at OSC to ~16 petabytes with better performance.

File paths

All project and scratch storage is available using the path /fs/ess/<project-code> for project space and /fs/scratch/<project-code> for scratch space.

Migrating storage

OSC have been migrating all current Project data and Scratch data to the new services since September 2022, and runs the final synchronization of the data during Oct 11 2022 downtime. ACLs and extended attributes for the data are also preserved after the migration. 

During December 13 2022 downtime, OSC cleaned the scratch directories for users who used to have scratch on both DDN and ESS storage (/fs/scratch/<project-code>and /fs/ess/scratch/<project-code>). All directories under /fs/ess/scratch/ points to /fs/scratch/ so they are essentially the same storage.

OSC have setup symlinks for the data on the storage so the compatibility for programs and job scripts is maintained. Please start to update your existing scripts to replace /fs/project/<project-code> with /fs/ess/<project-code> for project; and replace /fs/ess/scratch/<project-code> with  /fs/scratch/<project-code> for scratch.

Directories from OnDemand Files App

For users who used to have project space on the DDN storage, you will see /fs/ess/<project-code> instead of /fs/project/<project-code>. Please use the directory /fs/ess/<project-code> , which is your current project space location including all of your previous data on project. 

For users who used to have scratch on the ESS storage, you will see /fs/scratch/<project-code instead of /fs/ess/scratch/<project-code>). Please use the directory /fs/scratch/<project-code, which is your current scratch space location including all of your scratch data.