no way to compare when less than two revisions
Differences
This shows you the differences between two versions of the page.
Previous revision Next revision | |||
— | doku:vsc3_queue [2016/09/06 14:51] – [sbatch parameters] ir | ||
---|---|---|---|
Line 1: | Line 1: | ||
+ | ====== Queue | Partition setup on VSC-3 ====== | ||
+ | On VSC-3, the type of hardware and the quality of service (QOS) where the jobs run on may be selected. Nodes of the same type of hardware are grouped to partitions, the QOS defines the maximum run time of a job and the number and type of allocable nodes. | ||
+ | ===== Hardware types ===== | ||
+ | Three different types of compute nodes, nodes with 64 GB, 128 GB, and 256 GB, are available. | ||
+ | They are grouped into so-called ' | ||
+ | |||
+ | ^partition name^ description^ | ||
+ | | | | | ||
+ | |mem_0064 | default, nodes with 64 GB of memory | | ||
+ | |mem_0128 | nodes with 128 GB of memory| | ||
+ | |mem_0256 | nodes with 256 GB of memory| | ||
+ | | | | | ||
+ | |goodluck* | nodes with 64 GB of memory connected to faulty infiniband switches | | ||
+ | |||
+ | |||
+ | * //note//: about 50 infiniband switches are likely to fail at high oil temperatures (> 35 °C) due to overheating events in the past within the oil cooling environment of VSC-3. The nodes affected by the mentioned infiniband switches are grouped into the **goodluck** partition. Jobs running in this partition will not be added to the project accounting, i.e., jobs running there are **free of charge**. | ||
+ | |||
+ | ===== Quality of service (QOS) ===== | ||
+ | |||
+ | Access to node partitions is granted by the so-called quality of service (QOS). The QOSs constrain the number of allocatable nodes and limit job wall time. The naming scheme of the QOSs is: | ||
+ | < | ||
+ | |||
+ | The QOSs that are assigned to a specific user can be viewed with: | ||
+ | < | ||
+ | sacctmgr show user `id -u` withassoc format=user, | ||
+ | </ | ||
+ | |||
+ | ==== Run time limits ==== | ||
+ | |||
+ | |||
+ | ^ The QOS's hard run time limits ^ | | ||
+ | | | | | ||
+ | | normal_0064 / normal_0128 / normal_0256 | ||
+ | | idle_0064 / idle_0128 / idle_0256 | ||
+ | | private queues | ||
+ | | devel queue (up to 10 nodes available) | ||
+ | The QOS's run time limits can also be requested via the command | ||
+ | < | ||
+ | SLURM allows for setting a run time limit //below// the default QOS's run time limit. After the specified time is elapsed, the job is killed: | ||
+ | < | ||
+ | Acceptable time formats include " | ||
+ | |||
+ | === Backfilling === | ||
+ | |||
+ | Furthermore, | ||
+ | < | ||
+ | |||
+ | ==== sbatch parameters ==== | ||
+ | For submitting jobs, three parameters are important: | ||
+ | |||
+ | < | ||
+ | #SBATCH --partition=mem_xxxx | ||
+ | #SBATCH --qos=xxxxx_xxxx | ||
+ | #SBATCH --account=xxxxxx | ||
+ | </ | ||
+ | The core hours will be charged to the specified account. If not specified, the default account ('' | ||
+ | |||
+ | === ordinary projects === | ||
+ | |||
+ | For ordinary projects the QOSs are: | ||
+ | ^QOS name ^ gives access to partition ^description^ | ||
+ | | | | | ||
+ | |normal_0064 | mem_0064| default | | ||
+ | |normal_0128 | mem_0128| | | ||
+ | |normal_0256 | mem_0256| | | ||
+ | |goodluck | ||
+ | |devel_0128 | ||
+ | |||
+ | == examples == | ||
+ | < | ||
+ | #SBATCH --partition=mem_0128 | ||
+ | #SBATCH --qos=normal_0128 | ||
+ | #SBATCH --account=p7xxxx | ||
+ | </ | ||
+ | < | ||
+ | #SBATCH --partition=mem_0128 | ||
+ | #SBATCH --qos=devel_0128 | ||
+ | #SBATCH --account=p7xxxx | ||
+ | </ | ||
+ | * Note that partition, qos, and account have to fit together. | ||
+ | * If the account is not given, the default account ('' | ||
+ | * If partition and qos are not given, default values are mem_0064 and normal_0064. | ||
+ | |||
+ | === private nodes projects === | ||
+ | |||
+ | == example == | ||
+ | |||
+ | < | ||
+ | #SBATCH --partition=mem_xxxx | ||
+ | #SBATCH --qos=p7xxx_xxxx | ||
+ | #SBATCH --account=p7xxxx | ||
+ | </ | ||
+ | |||