Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
Next revisionBoth sides next revision
doku:gromacs [2023/05/17 15:54] – [Installations] msiegeldoku:gromacs [2023/06/14 11:31] – variants msiegel
Line 3: Line 3:
 Our recommendation: Follow these three steps to get the fastest program: Our recommendation: Follow these three steps to get the fastest program:
  
-  - Use the most **recent version** of GROMACS that we provide or build your own.+  - Use the **most recent version** of GROMACS that we provide or build your own.
   - Use the newest Hardware: use **1 GPU** on the partitions ''zen2_0256_a40x2'' or ''zen3_0512_a100x2'' on VSC-5.   - Use the newest Hardware: use **1 GPU** on the partitions ''zen2_0256_a40x2'' or ''zen3_0512_a100x2'' on VSC-5.
   - Do some **performance analysis** to decide if a single GPU Node (likely) or multiple CPU Nodes via MPI (unlikely) better suits your problem.   - Do some **performance analysis** to decide if a single GPU Node (likely) or multiple CPU Nodes via MPI (unlikely) better suits your problem.
Line 9: Line 9:
 In most cases it does not make sense to run on multiple GPU nodes with MPI; Whether using one or two GPUs per node. In most cases it does not make sense to run on multiple GPU nodes with MPI; Whether using one or two GPUs per node.
  
-===== GPU Partition =====+===== CPU or GPU Partition? =====
  
 First you have to decide on which hardware GROMACS should run, we call this a ''partition'', described in detail at [[doku:slurm | SLURM]]. On any login node, type ''sinfo'' to get a list of the available partitions, or take a look at [[doku:vsc4_queue]] and [[doku:vsc5_queue]]. We recommend the partitions ''zen2_0256_a40x2'' or ''zen3_0512_a100x2'' on VSC-5; they have plenty nodes available with 2 GPUs each. The partition has to be set in the batch script, see the example below. Be aware that each partition has different hardware, so choose the parameters accordingly. GROMACS decides mostly on its own how it wants to work, so don't be surprised if it ignores settings like environment variables. First you have to decide on which hardware GROMACS should run, we call this a ''partition'', described in detail at [[doku:slurm | SLURM]]. On any login node, type ''sinfo'' to get a list of the available partitions, or take a look at [[doku:vsc4_queue]] and [[doku:vsc5_queue]]. We recommend the partitions ''zen2_0256_a40x2'' or ''zen3_0512_a100x2'' on VSC-5; they have plenty nodes available with 2 GPUs each. The partition has to be set in the batch script, see the example below. Be aware that each partition has different hardware, so choose the parameters accordingly. GROMACS decides mostly on its own how it wants to work, so don't be surprised if it ignores settings like environment variables.
Line 15: Line 15:
 ===== Installations ===== ===== Installations =====
  
-We provide the following GROMACS installations:+Type ''spack find -l gromacs'' or ''module avail gromacs'' on ''cuda-zen''/''zen''/''skylake'' [[doku:spack-transition|spack trees]] on VSC-5/4. You can list available variants with [[doku:spack]]: ''spack find -l gromacs +cuda'' or ''spack find -l gromacs +mpi''.
  
-  * ''gromacs + cuda'': GPU Nodes, use the ''cuda-zen'' spack tree on VSC-5. +Because of the low efficiency of GROMACS on many nodes with many GPUs via MPI, we do not provide ''gromacs + cuda + mpi''.
-  * ''gromacs + mpi'': CPU only use ''zen''/''skylake'' spack trees on VSC-5/4.+
  
 +We provide the following GROMACS variants:
  
 +==== GPU but no MPI ====
  
-Type ''spack find -l gromacs'' or ''module avail gromacs'' on ''cuda-zen''/''zen''/''skylake'' spack trees on VSC-5/4. You can list available variants with spack: ''spack find -l gromacs +cuda'' or ''spack find -l gromacs +mpi''.+We recommend GPU Nodes, use the ''cuda-zen'' [[doku:spack-transition|spack tree]] on VSC-5:
  
-Because of the low efficiency of GROMACS on many nodes with many GPUs via MPI, we do not provide ''gromacs + cuda + mpi''. So the ''gromacs + cuda'' packages do not have MPI support, so there is no ''gmx_mpi'' binary, only 'gmx'.+**cuda-zen**: 
 +  * Gromacs +cuda ~mpi, all compiled with **GCC**
  
 +Since the ''gromacs + cuda'' packages do not have MPI support, there is no ''gmx_mpi'' binary, only ''gmx''.
 +
 +==== MPI but no GPU ====
 +
 +For Gromacs on CPU only but with MPI, use ''zen'' [[doku:spack-transition|spack tree]] on VSC-5 or ''skylake'' on VSC-4:
 +
 +**zen**:
 +  * Gromacs +openmpi +blas +lapack ~cuda, all compiled with **GCC**
 +  * Gromacs +openmpi +blas +lapack ~cuda, all compiled with **AOCC**
 +  * 
 +**skylake**:
 +  * Gromacs +**open**mpi +blas +lapack ~cuda, all compiled with **GCC**
 +  * Gromacs +**open**mpi +blas +lapack ~cuda, all compiled with **Intel**
 +  * Gromacs +**intel**mpi +blas +lapack ~cuda, all compiled with **GCC**
 +  * Gromacs +**intel**mpi +blas +lapack ~cuda, all compiled with **Intel**
 +
 +In some of these packages, there is no ''gmx'' binary, only ''gmx_mpi'', but this also works on one single node.
  
 ===== Batch Script ===== ===== Batch Script =====
Line 82: Line 101:
 benchmark various scenarios: benchmark various scenarios:
  
-  - a VSC users test case (??? atoms)+  - a VSC users test case (50,897 atoms)
   - R-143a in hexane (20,248 atoms) with very high output rate   - R-143a in hexane (20,248 atoms) with very high output rate
   - a short RNA piece with explicit water (31,889 atoms)   - a short RNA piece with explicit water (31,889 atoms)
  • doku/gromacs.txt
  • Last modified: 2023/11/23 12:27
  • by msiegel