check currently available openfoam modules/versions:

module avail openfoam*

load openfoam:

spack load -r openfoam-org@7 %intel@19.0.5.281     #load openfoam-org version 7 and additional packages
module load intel/19.0.5                           #loading compiler with spack not working, needs to be loaded separately

Example submission script:

#!/bin/sh
#SBATCH -J sim
#SBATCH -N 1
#SBATCH -A p71428
#SBATCH --qos p71428_0096 
#SBATCH --tasks-per-node=48 

module purge
spack load -r openfoam-org@7 %intel@19.0.5.281
module load intel/19.0.5

EXE=`which foamExec`

$EXE decomposePar -allRegions
mpirun -np 48 $EXE chtMultiRegionFoam -parallel
$EXE reconstructPar -newTimes -allRegions
$EXE rm -r processor*

load openfoam source file to have environment variables and be able to compile custom code:

source $WM_PROJECT_DIR/etc/bashrc

ODE-Solvers within openFoam-org versions should be used with the gcc-compiler.

Performance tests were done using different setups, depending on the used cluster:

VSC-1

• Intel C-compiler 11.1
• Qlogic MPI 0.1.0

VSC-2

• Intel C-compiler 12.1.3
• Intel MPI 4.0.3.008 / impi_intel-4.1.0.024

Compared to the gcc + openmpi Version a speed up of about 1/3 was found in case of 32 cores on VSC-1

Note: the runs denoted “VSC1 qlogic scaled” are the same as “VSC1 qlogic” but the runtimes were multiplied by a factor of 1.33 which is the nominal factor in clockspeed between VSC-1 and VSC-2

• mpich16: 16 cores per node were used for calculation
• mpich8: 8 cores per node were used for calculation

• mpich16: 16 cores per node were used for calculation