Both sides previous revision Previous revision Next revision | Previous revision Next revisionBoth sides next revision |
pandoc:introduction-to-vsc:09_special_hardware:accelerators [2021/05/19 11:31] – [Components on VSC-3+] goldenberg | pandoc:introduction-to-vsc:09_special_hardware:accelerators [2024/06/06 15:37] – [TOP500 List June 2020] goldenberg |
---|
| ====== GPUs available & how to use it ====== |
| |
====== Special hardware (GPUs, binfs) available & how to use it ====== | ===== TOP500 List June 2024 ===== |
| |
* Article written by Siegfried Höfinger (VSC Team) <html><br></html>(last update 2020-10-04 by sh). | |
| |
====== TOP500 List June 2020 ====== | |
| |
| |
<HTML> | |
<!--slide 1--> | |
<!--for nations flags see https://www.free-country-flags.com--> | |
</HTML> | |
^ Rank^Nation ^Machine ^ Performance^Accelerators ^ | ^ Rank^Nation ^Machine ^ Performance^Accelerators ^ |
| 1.|{{.:jp.png?0x24}} |Fugaku | 416 PFLOPs/s| | | | 1.|{{.:us.png?0x24}} | Frontier | 1206 PFLOPs/s| AMD Instinct MI250X | |
| 2.|{{.:us.png?0x24}} |Summit | 149 PFLOPs/s|<html><font color="navy"></html>NVIDIA V100<html></font></html> | | | 2.|{{.:us.png?0x24}} | Aurora | 1012 PFLOPs/s| Intel Data Center GPU Max | |
| 3.|{{.:us.png?0x24}} |Sierra | 95 PFLOPs/s|<html><font color="navy"></html>NVIDIA V100<html></font></html> | | | 3.|{{.:us.png?0x24}} | Eagle | 561 PFLOPs/s| NVIDIA H100 | |
| 4.|{{.:cn.png?0x24}} |Sunway TaihuLight | 93 PFLOPs/s| | | | 4.|{{.:jp.png?0x24}} | Fugaku | 442 PFLOPs/s| |
| 5.|{{.:cn.png?0x24}} |Tianhe-2A | 62 PFLOPs/s| | | | 5.| | LUMI | 379 PFLOPs/s| AMD Instinct MI250X | |
| 6.|{{.:it.png?0x24}} |HPC5 | 36 PFLOPs/s|<html><font color="navy"></html>NVIDIA V100<html></font></html> | | | 6.|{{.:ch.png?0x24}} | Alps | 270 PFLOPs/s| NVIDIA GH200 Superchip | |
| 7.|{{.:us.png?0x24}} |Selene | 28 PFLOPs/s|<html><font color="navy"></html>NVIDIA A100<html></font></html> | | | 7.|{{.:it.png?0x24}} | Leonardo | 241 PFLOPs/s| NVIDIA A100 SXM4 | |
| 8.|{{.:us.png?0x24}} |Frontera | 24 PFLOPs/s|<html><font color="navy"></html>NVIDIA RTX5000/V100<html></font></html> | | | 8.| | MareNostrum 5 ACC | 175 PFLOPs/s| NVIDIA H100 | |
| 9.|{{.:it.png?0x24}} |Marconi-100 | 22 PFLOPs/s|<html><font color="navy"></html>NVIDIA V100<html></font></html> | | | 9.|{{.:us.png?0x24}} | Summit | 148 PFLOPs/s| NVIDIA V100 | |
| 10.|{{.:ch.png?0x24}} |Piz Daint | 21 PFLOPs/s|<html><font color="navy"></html>NVIDIA P100<html></font></html> | | | 10.|{{.:us.png?0x24}} | Eos NVIDIA DGX | 121 PFLOPs/s| NVIDIA H100 | |
| |
| ===== Components on VSC-5 ===== |
| |
<HTML> | ^Model ^#cores ^Clock Freq (GHz)^Memory (GB)^Bandwidth (GB/s)^TDP (Watt)^FP32/FP64 (GFLOPs/s)^ |
<!--slide 2--> | |19x GeForce RTX-2080Ti n375-[001-019] - only in a special project | | | | | | |
</HTML> | |{{:pandoc:introduction-to-vsc:09_special_hardware:rtx-2080.jpg?nolink&200}} |4352|1.35 |11 |616 |250 |13450/420 | |
====== Components on VSC-3+ ====== | |45x2 nVidia A40 n306[6,7,8]-[001-019,001-019,001-007] | | | | | | | |
| |{{ :pandoc:introduction-to-vsc:09_special_hardware:a40.jpg?nolink&200|}} |10752 |1.305 |48 |696 |300 |37400/1169 | |
| |62x2 nVidia A100-40GB n307[1-4]-[001-015] | | | | | | |
| |{{ :pandoc:introduction-to-vsc:09_special_hardware:a100.jpg?nolink&200|}} |6912 |0.765 |40 |1555 |250 |19500/9700 | |
| |
^Model ^#cores ^Clock Freq (GHz)^Memory (GB)^Bandwidth (GB/s)^TDP (Watt)^FP32/FP64 (GFLOPs/s)^ | |
|<html><font color="navy"></html>42+51x GeForce GTX-1080 n37[1,2,3]-[001-006,001-022,001-026]<html></font></html>| | | | | | | | |
|{{.:nvidia-gtx-1080.jpg}} |2560 |1.61 |8 |320 |180 |8228/257 | | |
|<html><font color="navy"></html>4x Tesla k20m n372-02[4,5]<html></font></html> | | | | | | | | |
|{{.:nvidia-k20m.png}} |2496 |0.71 |5 |208 |195 |3520/1175 | | |
|<html><font color="navy"></html>1x Tesla V100 n372-023<html></font></html> | | | | | | | | |
|{{.:v100.png}} |5120/644|1.31 |32 |900 |250 |14000/7000 | | |
|<html><font color="navy"></html>19x GeForce RTX-2080Ti n375-[001-019]<html></font></html>| | | | | | | | |
|{{:pandoc:introduction-to-vsc:09_special_hardware:rtx-2080.jpg?nolink&200 |}} |4352|1.35 |11 |616 |250 |13450/420 | | |
|<html><font color="navy"></html>Avalable soon: 42x2 nVidia A40<html></font></html>| | | | | | | | |
| |
| ==== Working on GPU nodes Interactively ==== |
| |
| |
<HTML> | |
<!--slide 3--> | |
</HTML> | |
====== Working on GPU nodes ====== | |
| |
**Interactive mode** | **Interactive mode** |
| |
<code> | <code> |
1. VSC-3 > salloc -N 1 -p gpu_gtx1080single --qos gpu_gtx1080single | 1. VSC-5 > salloc -N 1 -p zen2_0256_a40x2 --qos zen2_0256_a40x2 --gres=gpu:2 |
| |
2. VSC-3 > squeue -u $USER | 2. VSC-5 > squeue -u $USER |
| |
3. VSC-3 > srun -n 1 hostname (...while still on the login node !) | 3. VSC-5 > srun -n 1 hostname (...while still on the login node !) |
| |
4. VSC-3 > ssh n372-012 (...or whatever else node had been assigned) | 4. VSC-5 > ssh n3066-012 (...or whatever else node had been assigned) |
| |
5. VSC-3 > module load cuda/9.1.85 | 5. VSC-5 > module load cuda/9.1.85 |
cd ~/examples/09_special_hardware/gpu_gtx1080/matrixMul | cd ~/examples/09_special_hardware/gpu_gtx1080/matrixMul |
nvcc ./matrixMul.cu | nvcc ./matrixMul.cu |
./a.out | ./a.out |
| |
6. VSC-3 > nvidia-smi | 6. VSC-5 > nvidia-smi |
| |
7. VSC-3 > /opt/sw/x86_64/glibc-2.17/ivybridge-ep/cuda/9.1.85/NVIDIA_CUDA-9.1_Samples/1_Utilities/deviceQuery/deviceQuery | 7. VSC-5 > /opt/sw/x86_64/glibc-2.17/ivybridge-ep/cuda/9.1.85/NVIDIA_CUDA-9.1_Samples/1_Utilities/deviceQuery/deviceQuery |
</code> | </code> |
<HTML> | <HTML> |
<!--slide 4--> | <!--slide 4--> |
</HTML> | </HTML> |
====== Working on GPU nodes cont. ====== | |
| ===== Working on GPU using SLURM ===== |
| |
**SLURM submission** gpu_test.scrpt | **SLURM submission** gpu_test.scrpt |
# usage: sbatch ./gpu_test.scrpt | # usage: sbatch ./gpu_test.scrpt |
# | # |
#SBATCH -J gtx1080 | #SBATCH -J A40 |
#SBATCH -N 1 | #SBATCH -N 1 #use -N only if you use both GPUs on the nodes, otherwise leave this line out |
#SBATCH --partition gpu_gtx1080single | #SBATCH --partition zen2_0256_a40x2 |
#SBATCH --qos gpu_gtx1080single | #SBATCH --qos zen2_0256_a40x2 |
| #SBATCH --gres=gpu:2 #or --gres=gpu:1 if you only want to use half a node |
| |
module purge | module purge |
/opt/sw/x86_64/glibc-2.17/ivybridge-ep/cuda/9.1.85/NVIDIA_CUDA-9.1_Samples/1_Utilities/deviceQuery/deviceQuery | /opt/sw/x86_64/glibc-2.17/ivybridge-ep/cuda/9.1.85/NVIDIA_CUDA-9.1_Samples/1_Utilities/deviceQuery/deviceQuery |
</code> | </code> |
<html><font color="navy"></html>**Exercise/Example/Problem:**<html></font></html> <html><br/></html> Using interactive mode or batch submission, figure out whether we have ECC enabled on GPUs of type gtx1080 ? | |
| |
<HTML> | <HTML> |
</HTML> | </HTML> |
| |
====== Real-World Example, AMBER-16 ====== | ===== Real-World Example, AMBER-16 ===== |
| |
^ Performance^Power Efficiency ^ | ^ Performance^Power Efficiency ^ |
| {{.:amber16.perf.png}}|{{.:amber16.powereff.png}} | | | {{.:amber16.perf.png}}|{{.:amber16.powereff.png}} | |
| |