User documentation
General
Software & Services
under construction
Queuing system
Systems
VSC 4
VSC 3
VSC 2
Decomissioned
VSC 1
Decomissioned
Decomissioned
Decomissioned
On VSC-3, the type of hardware and the quality of service (QOS) where the jobs run on may be selected. Nodes of the same type of hardware are grouped to partitions, the QOS defines the maximum run time of a job and the number and type of allocable nodes.
Three different types of compute nodes, nodes with 64 GB, 128 GB, and 256 GB, GPU nodes, and Knights Landing (KNL) are available.
On VSC-3, the hardware is grouped into so-called ➠ partitions:
partition name | description |
---|---|
mem_0064 | default, nodes with 64 GB of memory |
mem_0128 | nodes with 128 GB of memory |
mem_0256 | nodes with 256 GB of memory |
knl | Knights Landing (KNL) |
gpu | GPU nodes |
Access to node partitions is granted by the so-called ➠ quality of service (QOS). The QOSs constrain the number of allocatable nodes and limit job wall time. The naming scheme of the QOSs is: <project_type>_<memoryConfig>
The QOSs that are assigned to a specific user can be viewed with:
sacctmgr show user `id -u` withassoc format=user,defaultaccount,account,qos%40s,defaultqos%20s
Have a look on the ➠ available QOSs on VSC-3.
The QOS's hard run time limits | |
---|---|
normal_0064 / normal_0128 / normal_0256 | 72h (3 days) |
idle_0064 / idle_0128 / idle_0256 | 24h (1 day) |
private queues p….._0… | 240h (10 days) |
devel queue (up to 10 nodes available) | 10min |
The QOS's run time limits can also be requested via the command
sacctmgr show qos format=name%20s,priority,grpnodes,maxwall,description%40s
SLURM allows for setting a run time limit below the default QOS's run time limit. After the specified time is elapsed, the job is killed:
#SBATCH --time=<time>
Acceptable time formats include “minutes”, “minutes:seconds”, “hours:minutes:seconds”, “days-hours”, “days-hours:minutes” and “days-hours:minutes:seconds”.
Furthermore, it is possible to set a minimum time limit on the job allocation. When jobs with different demands of resources are scheduled, it is most likely that not all nodes can be filled. Imagine a job requesting more resources than are currently free and, thus, cannot start. Since it has the highest priority, all other jobs would need to wait. Backfilling means the process to fill up those idle nodes with jobs fitting the unused time gap. This permits the time limited job to be scheduled earlier than jobs with higher piority. It is highly encouraged to guess this minimum time where possible because it also contributes to a better cluster usage:
#SBATCH --time-min=<time>
For submitting jobs, three parameters are important:
#SBATCH --partition=mem_xxxx #SBATCH --qos=xxxxx_xxxx #SBATCH --account=xxxxxx
The core hours will be charged to the specified account. If not specified, the default account (sacctmgr show user `id -u` withassoc format=defaultaccount
) will be used.
For ordinary projects the QOSs are:
QOS name | gives access to partition | description |
---|---|---|
normal_0064 | mem_0064 | default |
normal_0128 | mem_0128 | |
normal_0256 | mem_0256 | |
devel_0128 | mem_0128 | for development purposes only 10 min & 10 nodes |
#SBATCH --partition=mem_0128 #SBATCH --qos=normal_0128 #SBATCH --account=p7xxxx
#SBATCH --partition=mem_0128 #SBATCH --qos=devel_0128 #SBATCH --account=p7xxxx
sacctmgr show user `id -u` withassoc format=defaultaccount
) will be used.#SBATCH --partition=mem_xxxx #SBATCH --qos=p7xxx_xxxx #SBATCH --account=p7xxxx