VSC-4 and VSC-5 provide two unified facilities for data: the high-performance IBM Storage Scale Parallel Filesystem and a node-local flash storage. The parallel filesystem is the same on both clusters, so projects that exist on both clusters can access the same data. They are accessible under:

• IBM Spectrum Scale:
  • $HOME expands to /home/fs<project>/<username>,
  • $DATA expands to /gpfs/data/fs<project>/<username>
• Local SSD
  • /local

$HOME is the location of the user UNIX home directory. It can be accessed from login and compute nodes. $HOME should be used for storing/compiling code - data for which high concurrent job throughput and support for large file sizes is not required. Quota on $HOME is 100GB and 10^6 number of files and the size can not be extended (number of files can be).

$DATA is a tiered file system containing 500TB flash and around 5PB of HDD storage. Quota on $DATA is 10TB and 10^6 number of files and can be extended up to 100TB. If even more size is needed, specific arrangements can be made.

/local is a locally mounted NVMe disk. The size is about 450GB on VSC-4 and 1.8TB on VSC-5. Data retention is not guaranteed, al data is lost on a reboot, so they need to be transferred after use.

We operate a backup for disaster recovery on two physical sites with IBM Jaguar tape libraries. In the case of $DATA, this backup can be toggled off and on in the project management page. Restore of lost or deleted files is not possible. Backup of user files is solely the responsibility of each user.