This version is outdated by a newer approved version.
This version (2023/03/14 13:06) is a draft.
Approvals: 0/1
![Diff](/lib/images/diff.png)
Approvals: 0/1
This is an old revision of the document!
GPUs available & how to use it
- Article written by Siegfried Höfinger (VSC Team) <html><br></html>(last update 2020-10-04 by sh).
TOP500 List June 2020
<HTML> <!–slide 1–> <!–for nations flags see https://www.free-country-flags.com–> </HTML>
<HTML> <!–slide 2–> </HTML>
Components on VSC
GPUs on VSC-3 - now decommissioned
GPUs on VSC-5
Model | #cores | Clock Freq (GHz) | Memory (GB) | Bandwidth (GB/s) | TDP (Watt) | FP32/FP64 (GFLOPs/s) |
---|---|---|---|---|---|---|
<html><font color=“navy”></html>19x GeForce RTX-2080Ti n375-[001-019]<html></font></html> | ||||||
![]() | 4352 | 1.35 | 11 | 616 | 250 | 13450/420 |
<html><font color=“navy”></html>45×2 nVidia A40 n306[6,7,8]-[001-019,001-019,001-007]<html></font></html> | ||||||
![]() | 10752 | 1.305 | 48 | 696 | 300 | 37400/1169 |
<html><font color=“navy”></html>60×2 nVidia A100-40GB n307[1-4]-[001-015]<html></font></html> | ||||||
![]() | 6912 | 0.765 | 40 | 1555 | 250 | 19500/9700 |
<HTML> <!–slide 3–> </HTML>
Working on GPU nodes Interactively
Interactive mode
1. VSC-5 > salloc -N 1 -p zen2_0256_a40x2 --qos zen2_0256_a40x2 --gres=gpu:2 2. VSC-5 > squeue -u $USER 3. VSC-5 > srun -n 1 hostname (...while still on the login node !) 4. VSC-5 > ssh n3066-012 (...or whatever else node had been assigned) 5. VSC-5 > module load cuda/9.1.85 cd ~/examples/09_special_hardware/gpu_gtx1080/matrixMul nvcc ./matrixMul.cu ./a.out cd ~/examples/09_special_hardware/gpu_gtx1080/matrixMulCUBLAS nvcc matrixMulCUBLAS.cu -lcublas ./a.out 6. VSC-5 > nvidia-smi 7. VSC-5 > /opt/sw/x86_64/glibc-2.17/ivybridge-ep/cuda/9.1.85/NVIDIA_CUDA-9.1_Samples/1_Utilities/deviceQuery/deviceQuery
<HTML> <!–slide 4–> </HTML>
Working on GPU using SLURM
SLURM submission gpu_test.scrpt
#!/bin/bash # # usage: sbatch ./gpu_test.scrpt # #SBATCH -J A40 #SBATCH -N 1 #SBATCH --partition zen2_0256_a40x2 #SBATCH --qos zen2_0256_a40x2 module purge module load cuda/9.1.85 nvidia-smi /opt/sw/x86_64/glibc-2.17/ivybridge-ep/cuda/9.1.85/NVIDIA_CUDA-9.1_Samples/1_Utilities/deviceQuery/deviceQuery
<html><font color=“navy”></html>Exercise/Example/Problem:<html></font></html> <html><br/></html> Using interactive mode or batch submission, figure out whether we have ECC enabled on GPUs of type gtx1080 ?
<HTML> <!–slide 5–> </HTML>