Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
doku:binf_nodes [2017/07/06 12:17]
sreinwal
doku:binf_nodes [2017/09/01 13:51] (current)
ir [Quotas / How to get more diskspace]
Line 37: Line 37:
 </code> </code>
  
-Please note that one has to include the --mount=/binfs or --mount=/binfl part, otherwise the quota for the $GLOBAL filesystem will be shown.  +Please note that one has to include the --mount=/binfs or --mount=/binfl part, otherwise the quota for the ''$GLOBAL'' filesystem will be shown. 
- +
-Additional info on BeeGFS quotas can be found [[https://wiki.vsc.ac.at/doku.php?id=doku:vsc3_global_quotas|here]].+
  
 +For additional info refer to [[https://wiki.vsc.ac.at/doku.php?id=doku:vsc3_global_quotas|BeeGFS quotas]].
  
 ==== Available filesystems ==== ==== Available filesystems ====
  
-The binf nodes export 2 mountpoints which are $BINFS (/binfs) and $BINFL (/binfl). The BINFS filesystem is more than twice as fast as the BINFL filesystem and even a magnitude faster then the $GLOBAL filesystem. If you need really high IO rates you should use the BINFS filesystem. +The binf nodes export 2 mountpoints which are ''$BINFS'' (/binfs) and ''$BINFL'' (/binfl). The BINFS filesystem is more than twice as fast as the BINFL filesystem and even a magnitude faster then the ''$GLOBAL'' filesystem. If you need really high IO rates you should use the BINFS filesystem. 
  
-If you need much space then you should use the BINFL filesystem, which is 10 times larger than BINFS. +If you need much space then you should use the ''$BINFL'' filesystem, which is 10 times larger than ''$BINFS''
  
-Please keep in mind that BINFS has no redundancy and is seen as fast scratch space. A hardware defect can (and most probably will, and already has) lead to a situation where the whole file systems gets reinitialised, which means the wiping of the whole data on the parallel filesystem. So if you can't live with that, you should use the BINFL file system which has a bit more redundancy.+Please keep in mind that ''$BINFS'' has no redundancy and is seen as fast scratch space. A hardware defect can (and most probably will, and already has) lead to a situation where the whole file systems gets reinitialised, which means the wiping of the whole data on the parallel filesystem. So if you can't live with that, you should use the ''$BINFL'' file system which has a bit more redundancy.
  
 ==== Submitting Jobs ==== ==== Submitting Jobs ====
Line 69: Line 68:
 to get in the priority queue, which has a maximum runtime of 72 hours.  to get in the priority queue, which has a maximum runtime of 72 hours. 
  
-If you just want to utilise the filesystems you can use the $BINFS/$BINFL environment variables in your scripts.+If you just want to utilise the filesystems you can use the ''$BINFS/$BINFL'' environment variables in your scripts.
  
 === Using high memory nodes === === Using high memory nodes ===
Line 86: Line 85:
 ==== Pitfalls ==== ==== Pitfalls ====
  
-  * Distributed file systems do not perform well when it comes to random I/O. If you need to do random I/O it would be best to first copy your data into the main memory of the node that is working on the data and do the random I/O in the main memory. To accomplish this you could either use the $TMP filesystem or use the [[http://www.manpages.info/linux/mmap.2.html|mmap system call]].+  * Distributed file systems do not perform well when it comes to random I/O. If you need to do random I/O it would be best to first copy your data into the main memory of the node that is working on the data and do the random I/O in the main memory. To accomplish this you could either use the ''$TMP'' filesystem or use the [[http://www.manpages.info/linux/mmap.2.html|mmap system call]].
  
  
  
  • doku/binf_nodes.1499343455.txt.gz
  • Last modified: 2017/07/06 12:17
  • by sreinwal