Both sides previous revision Previous revision Next revision | Previous revision Next revisionBoth sides next revision |
doku:vsc4_storage [2021/06/09 13:51] – mhickel | doku:vsc4_storage [2023/10/23 10:55] – [Usage '/local'] goldenberg |
---|
===== VSC-4 Storage ===== | ===== VSC Storage ===== |
| |
VSC-4 provides three facilities for persisting data: the high-performance IBM Spectrum Scale Parallel Filesystem, BeeGFS Parallel Filesystem (former Fraunhofer Parallel Filesystem, FhGFS) and a node-local SSD. They are accessible under: | VSC-4 and VSC-4 provide two unified facilities for data: the high-performance IBM Storage Scale Parallel Filesystem and a node-local flash storage. The parallel filesystem is the same on both clusters, so projects that exist on both clusters can access the same data. |
| They are accessible under: |
| |
* **IBM Spectrum Scale**: | * **IBM Spectrum Scale**: |
* ''$HOME'' expands to /home/fs<project>/<username>, | * ''$HOME'' expands to /home/fs<project>/<username>, |
* ''$DATA'' expands to /gpfs/data/fs<project>/<username> | * ''$DATA'' expands to /gpfs/data/fs<project>/<username> |
* **BeeGFS (former FhGFS)**: | * **Local SSD** |
* ''$BINFL'' expands to /binfl/lv<project>/<username>, | * ''/local'' |
* ''$BINFS'' expands to /binfs/lv<project>/<username> | |
* **Local SSD** /local | ==== Usage ''$HOME'' ==== |
| |
| ''$HOME'' is the location of the user UNIX home directory. It can be accessed from login and compute nodes. ''$HOME'' should be used for storing/compiling code - data for which high concurrent job throughput and support for large file sizes is not required. |
| Quota on ''$HOME'' is 100GB and 10^6 number of files and the size **can not** be extended (number of files can be). |
| |
| ==== Usage ''$DATA'' ==== |
| |
| ''$DATA'' is a tiered file system containing 500TB flash and around 5PB of HDD storage. |
| Quota on ''$DATA'' is 10TB and 10^6 number of files and can be extended up to 100TB. If even more size is needed, specific arrangements can be made. |
| |
| ==== Usage '/local' ==== |
| |
| /local is a locally mounted NVMe disk. The size is about 450GB on VSC-4 and 1.8TB on VSC-5. Data retention is not guaranteed, al data is lost on a reboot, so they need to be transferred after use. |
| |
| ==== Backup Policy ==== |
| |
| Backup of user files independent of location is solely the responsibility of each user. |
| |