Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
doku:vsc3_storage [2021/05/13 18:05]
goldenberg [Backup Policy]
doku:vsc3_storage [2021/08/23 08:52] (current)
goldenberg [Quotas]
Line 24: Line 24:
  
  
-==== Scratch Space Usage: ''\$GLOBAL'' and ''\$SCRATCH'' ==== 
  
-**This is decommissioned and no longer available** 
- 
-The [[http://www.beegfs.com|BeeGFS]] parallel file system (former FhGFS) on VSC-3 provides a large (initially approx. 0.5 PB) scratch space. The environment variable ''$GLOBAL'' expands to: 
- 
-<code> 
-$ echo $GLOBAL 
-/global/lv70999/username 
-</code> 
- 
-The directory is writeable as user and readable by the group members. It is advisable to make use of these directories in particular for jobs with heavy I/O operations. In addition it will reduce the load on the fileserver holding the $HOME directories. 
- 
-The BeeGFS (former Fraunhofer parallel file system) is **shared by all users and by all nodes**. 
-Single jobs producing heavy load (>>1000 requests per second) have been observed to reduce responsiveness for all jobs and all users. 
- 
-Lifetime of data is limited, see table below. 
- 
- 
-=== Per-node Scratch Directories ''$SCRATCH'' === 
- 
-Local scratch directories on each node are provided as a link to the BeeGFS parallel file system and can thus be viewed also via the login nodes as '''/fhgfs/nodelocal/rXXnXX/''' 
-The parallel file system (and thus the performance) is identical between $SCRATCH and $GLOBAL. 
-The variable ''$SCRATCH'' expands to: 
-<code> 
-$ echo $SCRATCH 
-/scratch 
-</code> 
-These directories are purged after job execution. 
  
 ==== Usage Local Scratch RAM Disk $TMPDIR ==== ==== Usage Local Scratch RAM Disk $TMPDIR ====
Line 63: Line 35:
 </code> </code>
  
-\\ 
-==== Comparison of scratch directories ==== 
- 
-|                      || ''$GLOBAL''                        || ''$SCRATCH''               || ''$TMPDIR''    || 
-| Recommended file size || large                         || large                  || small        || 
-| Lifetime             || files older than 90 days deleted if $GLOBAL space is running low     || job                    || job                     || 
-| Size                 || x00 TB (for all users)         || x00 TB (for all users) || a few GB (within memory) ||  
-| Scaling              || does not fit very large number of small file IO || does not fit very large number of small file IO  || very good (local) || 
-| Visibility           || global                         || node (see above)      || node                    ||  
-| Recommended usage     || large files, available temporarily after job life || large files    || many small files (>1000, or many seek-operations within a file ||  
 \\ \\
  
Line 80: Line 42:
  
 Storage extensions can be requested through [[https://service.vsc.tuwien.ac.at/|Vergabeassistent]] at Extensions - Storage. Storage extensions can be requested through [[https://service.vsc.tuwien.ac.at/|Vergabeassistent]] at Extensions - Storage.
- 
-Since 2017-06-02 quotas are enforced for the ''$GLOBAL'' filesystem. Additional information can be found [[vsc3_global_quotas|here]] 
  
 ==== Fair Use ==== ==== Fair Use ====
  • doku/vsc3_storage.1620929126.txt.gz
  • Last modified: 2021/05/13 18:05
  • by goldenberg