Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
doku:memory [2014/07/30 10:01] – irdoku:memory [Unknown date] (current) – external edit (Unknown date) 127.0.0.1
Line 1: Line 1:
 ====== Memory intensive Jobs ====== ====== Memory intensive Jobs ======
  
-Jobs needing more than 2 GB per process have several alternatives. They might use+Jobs requiring more than 2 GB per process have several alternatives. They might use
   - a node with more memory,   - a node with more memory,
   -  a parallel environment with fewer processes  per node,   -  a parallel environment with fewer processes  per node,
Line 23: Line 23:
 several nodes on the VSC-2 have 64 GB or more instead of the 32 GB of the standard nodes. several nodes on the VSC-2 have 64 GB or more instead of the 32 GB of the standard nodes.
 To use one of these nodes, just add '''-l mem_free=50G''' or '''-l mem_free=100G''' to ''qsub''. To use one of these nodes, just add '''-l mem_free=50G''' or '''-l mem_free=100G''' to ''qsub''.
-[[vsc2#node_types | Additional details]] 
  
-==== ad 2. parallel environment with fewer processes  per node ====+See also [[doku:memory_running_jobs|memory usage of running jobs]]!
  
-Jobs using more than 2 GB per core can be executed on one of the parallel environments +==== ad 2. parallel environment with fewer processes  per node ====
-  * mpich8: 4 GB per core +
-  * mpich4: 8 GB per core +
-  * mpich2: 16 GB per core +
-  * mpich1: 32 GB per core+
  
-Please keep in mind that on VSC-2 jobs are node-exclusive and therefore your contingent of CPU-hours will be computed by full nodes and are therefore significantly more expensive. +   *[[doku:ompmpi|Hybrid OpenMP/MPI jobs ]]
-For example, <code>qsub -pe mpich2 64 my_job.sh</code> requests 64 cores, each with 16 GB: this is fulfilled by 32 nodes: each hour of this job costs 32*16=512 CPU-hours.+
  
-The variable NSLOTS_REDUCED is set to the number of cores requested, whereas the variable NSLOTS is set to the number of cores allocated in the queueing system, which corresponds to the cost calculation of the previous paragraph. 
-Replace your calls to mpirun accordingly to<code>mpirun -n $NSLOTS_REDUCED ... </code> where applicable! 
  
 ==== ad 3. increased virtual memory ==== ==== ad 3. increased virtual memory ====
  
-Some programs allocate more memory than they use. This was especially true in old FORTRAN 77 programs, which had to decide at compile time how much memory will be used. These programs are allowed to allocate 50% more memory than available by '''#$ -l overcommit_mem=true''' in the job script. Unfortunately it might happen that the whole node crashes, reboots and leaves the queuing system, so please use this option wisely!+Some programs allocate more memory than they use. This was especially true in old FORTRAN 77 programs, which had to decide at compile time how much memory will be used. These programs are allowed to allocate 50% more memory than available by ''#$ -l overcommit_mem=true'' in the job script. Unfortunately it might happen that the whole node crashes, reboots and leaves the queuing system, so please use this option wisely!
  
 ==== ad 4. swap space (still experimental) ==== ==== ad 4. swap space (still experimental) ====
  
 A novel feature of the VSC-2 is remote swap space (implemented using 'SCSI RDMA Protocol', SRP), which is used by specifying '''#$ -l swapsize_GB=32''' or a multiple of 32. Each node of a job gets the same amount of swap space. A novel feature of the VSC-2 is remote swap space (implemented using 'SCSI RDMA Protocol', SRP), which is used by specifying '''#$ -l swapsize_GB=32''' or a multiple of 32. Each node of a job gets the same amount of swap space.
  • doku/memory.1406714487.txt.gz
  • Last modified: 2014/07/30 10:01
  • by ir