Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
doku:slurm [2023/03/14 12:34] – [Job submission] goldenberg | doku:slurm [2024/07/11 09:05] (current) – grokyta | ||
---|---|---|---|
Line 15: | Line 15: | ||
On VSC-4 and VSC-5, spack is used to install and provide modules, see [[doku: | On VSC-4 and VSC-5, spack is used to install and provide modules, see [[doku: | ||
+ | In order to set environment variables needed for a specific application, | ||
+ | * '' | ||
+ | * '' | ||
+ | * '' | ||
+ | * '' | ||
+ | * '' | ||
+ | * '' | ||
+ | == Note: == | ||
+ | |||
+ | - **< | ||
+ | - a list of '' | ||
+ | |||
+ | When all required/ | ||
==== Node configuration - hyperthreading ==== | ==== Node configuration - hyperthreading ==== | ||
Line 29: | Line 42: | ||
ThreadsPerCore=2 | ThreadsPerCore=2 | ||
</ | </ | ||
- | This reflects the fact that <html> < | + | This reflects the fact that <color #cc3300> hyperthreading </color> is activated on all compute nodes and <color #cc3300> 96 cores on VSC4 and 256 cores on VSC5 </color> may be utilized on each node. |
In the batch script hyperthreading is selected by adding the line | In the batch script hyperthreading is selected by adding the line | ||
- | < | + | < |
- | #SBATCH --ntasks-per-core=2 | + | |
</ | </ | ||
which allows for 2 tasks per core. | which allows for 2 tasks per core. | ||
Line 41: | Line 53: | ||
==== Node allocation policy ==== | ==== Node allocation policy ==== | ||
- | On VSC-4 there are a set of nodes which accept jobs that do not require entire nodes (anythong | + | On VSC-4 & VSC-5 there is a set of nodes that accept jobs that do not require entire |
- | On VSC-5 that feature | + | |
Line 50: | Line 62: | ||
Depending on the demands of a certain application, | Depending on the demands of a certain application, | ||
- | [[doku:vsc3_queue|partition (grouping hardware according its type) and | + | [[doku:vsc5_queue|partition (grouping hardware according its type) and |
quality of service (QOS; defining the run time etc.)]] can be selected. | quality of service (QOS; defining the run time etc.)]] can be selected. | ||
- | Additionally, | + | Additionally, |
==== The job submission script==== | ==== The job submission script==== | ||
It is recommended to write the job script using a [[doku: | It is recommended to write the job script using a [[doku: | ||
- | Editors in //Windows// may add additional invisible characters to the job file which render it unreadable and, thus, it is not executed. | + | Editors in //Windows// may add additional invisible characters to the job file which render it unreadable and, thus, it cannot be not executed. |
Assume a submission script '' | Assume a submission script '' | ||
Line 76: | Line 88: | ||
</ | </ | ||
* **-J** | * **-J** | ||
- | * **-N** | + | * **-N** |
* **-n, --ntasks=< | * **-n, --ntasks=< | ||
* **--ntasks-per-node** | * **--ntasks-per-node** | ||
Line 94: | Line 106: | ||
# is obtained from the previous command | # is obtained from the previous command | ||
</ | </ | ||
- | |||
- | |||
- | |||
- | |||
- | |||
- | ====A word on srun and mpirun:==== | ||
- | Currently (27th March 2015), **srun** only works when the application uses **intel mpi** and is compiled with the **intel compiler**. We will provide compatible versions of MVAPICH2 and OpenMPI in the near future. | ||
- | At the moment, it is recommended to use **mpirun** in case of MVAPICH2 and OpenMPI. | ||
- | |||
==== Hybrid MPI/OMP: ==== | ==== Hybrid MPI/OMP: ==== | ||
Line 145: | Line 148: | ||
</ | </ | ||
See also the [[https:// | See also the [[https:// | ||
+ | |||
Line 157: | Line 161: | ||
#SBATCH -J chk | #SBATCH -J chk | ||
#SBATCH -N 4 | #SBATCH -N 4 | ||
- | #SBATCH --ntasks-per-node=16 | + | #SBATCH --ntasks-per-node=48 |
#SBATCH --ntasks-per-core=1 | #SBATCH --ntasks-per-core=1 | ||
Line 163: | Line 167: | ||
scontrol show hostnames $SLURM_NODELIST | scontrol show hostnames $SLURM_NODELIST | ||
- | srun -l -N2 -r0 -n32 job1.scrpt & | + | srun -l -N2 -r0 -n96 job1.scrpt & |
- | srun -l -N2 -r2 -n32 job2.scrpt & | + | srun -l -N2 -r2 -n96 job2.scrpt & |
wait | wait | ||
- | srun -l -N2 -r2 -n32 job3.scrpt & | + | srun -l -N2 -r2 -n96 job3.scrpt & |
- | srun -l -N2 -r0 -n32 job4.scrpt & | + | srun -l -N2 -r0 -n96 job4.scrpt & |
wait | wait | ||
Line 276: | Line 280: | ||
#SBATCH -J par # job name | #SBATCH -J par # job name | ||
#SBATCH -N 2 # number of nodes=2 | #SBATCH -N 2 # number of nodes=2 | ||
- | #SBATCH --ntasks-per-node=16 # uses all cpus of one node | + | #SBATCH --ntasks-per-node=48 # uses all cpus of one node |
#SBATCH --ntasks-per-core=1 | #SBATCH --ntasks-per-core=1 | ||
#SBATCH --threads-per-core=1 | #SBATCH --threads-per-core=1 | ||
Line 286: | Line 290: | ||
rm machines_tmp | rm machines_tmp | ||
- | tasks_per_node=16 # change number accordingly | + | tasks_per_node=48 # change number accordingly |
nodes=2 | nodes=2 | ||
for ((line=1; line< | for ((line=1; line< | ||
Line 338: | Line 342: | ||
- continue at 2. for further dependent jobs | - continue at 2. for further dependent jobs | ||
- | ===== Licenses ===== | ||
- | Software, that uses a license server, has to be specified upon job submission. A list of all available licensed software for your user can be shown by using the command: | ||
- | |||
- | < | ||
- | slic | ||
- | </ | ||
- | |||
- | Within the job script add the flags as shown with ' | ||
- | |||
- | < | ||
- | #SBATCH -L matlab@vsc, | ||
- | </ | ||
===== Prolog Error Codes ===== | ===== Prolog Error Codes ===== |