Queues

The majority of jobs use the standard queue 'all.q' with a maximum run time of 3 days (72 hours).

All nodes are configured equivalently, only a few nodes have different amounts of memory. VSC-2 has

• 1334 nodes with 32GB main memory
• 8 nodes with 64 GB main memory
• 8 nodes with 128 GB main memory
• 1 node with 512 GB main memory and 64 cores (not in 'all.q', but in 'highmem.q')

Jobs are scheduled by default to request nodes with at least 27 GB free memory. To override this default on the command line or in the job script you may specify:

• default: allows scheduling on 32 GB, 64 GB, or 128 GB nodes
• request scheduling on 64 GB or 128 GB nodes, only:
  • '-l mem_free=50G' (on the command line) or
  • '#$ -l mem_free=50G' (in the script),
• request scheduling on 128 GB nodes, only:
  • '-l mem_free=100G' (command line) or
  • '#$ -l mem_free=100G' (job script).

In order to avoid jobs with low memory requirements on nodes with 64 or 128 GB, priority adjustments are made in the queue.

A queue where jobs will be allowed to run a maximum of 7 days is available on VSC-2. The limit on the number of slots per job is 128 and the maximum number of allocatable slots per user at one time is 768. A total of 4096 slots are available for long jobs. All nodes of this queue have 32GB main memory. Use this queue by specifying it explicitly in your job script:

#$ -q long.q

or submit your job with

 qsub -q long.q <job_file>

Due to higher memory requests from some users, one node with 512 GB memory and 64 cores is available in the queue 'highmem.q'. The four processors utilized are AMD Opteron 6274 with 2.2GHz and 16 cores each. These nodes show a sustained performance of about 400 GFlop/s, which compares to about four standard nodes of the VSC-2.

Due to the special memory requirements of jobs in this queue, jobs are granted exclusive access. 64 slots are accounted for, even if the job does not make efficient use of all 64 cores. Make sure to adapt your job script to pin processes to cores

export I_MPI_PIN_PROCESSOR_LIST=0-63

if applicable.

The run time limit is 3 days (72 hours).

Programs which work in the 'all.q' and the 'long.q' run without modifications on these nodes, too. Intel compilers and Intel MPI show good behaviour on the 'highmem.q' queue.

Please use this node only for jobs with memory requirements of more than 64 GB!