• environment: global variables available to current and all sub-shells (env/set )
• http://lmod.readthedocs.io/en/latest/010_user.html
• module: all modifications (set and prepend to variables) in the user environment necessary for using a piece of software
• handles additional software, which is not installed to default system paths
• makes different versions of one piece of software available
• prerequisites: loading a module may require loading other modules before (handled by module dependency)

module avail
module list
module purge
module load <module>
module unload <module>
module rm <module>
module swap <module 1> <module 2>
module show <module>
module whatis <module>
module help <module>
module spider
module help

Modules set paths of environment variables like PATH, LD_LIBRARY_PATH, LIBRARY_PATH, etc. This allows you to select a specific compiler, libraries, or other software versions without having to specify the full path to the files or binaries.

module avail

• shows available modules.
• searches paths given by MODULEPATH
• own modules can be added by extending MODULEPATH
• If multiple modules with the same name/version are present in different directories, the first found by MODULEPATH will be used.

Software packages often have dependecies on specific versions of a compiler and/or on libraries.
E.g. the software scalapack has dependencies on the selected MPI and compiler version:

module purge
module avail

------------------ /opt/ohpc/pub/modulefiles ---------------------------------------------
   EasyBuild/3.4.1    autotools    clustershell/1.8    cmake/3.9.2    gnu/5.4.0    gnu7/7.2.0
   hwloc/1.11.8  llvm4/4.0.1    ohpc    papi/5.5.1    pmix/1.2.3    prun/1.2    singularity/2.4
   valgrind/3.13.0

module load gnu7/7.2.0 
module avail

------------------ /opt/ohpc/pub/moduledeps/gnu7 ------------------------------------------
   R/3.4.2    gsl/2.4    hdf5/1.10.1    metis/5.1.0    mpich/3.2    mvapich2/2.2    numpy/1.13.1
   ocr/1.0.1  openblas/0.2.20    openmpi/1.10.7    openmpi3/3.0.0    pdtoolkit/3.24    plasma/2.8.0
   scotch/6.0.4    superlu/5.2.1

------------------ /opt/ohpc/pub/modulefiles ----------------------------------------------
   EasyBuild/3.4.1    autotools    clustershell/1.8    cmake/3.9.2    gnu/5.4.0    gnu7/7.2.0 (L)
   hwloc/1.11.8  llvm4/4.0.1    ohpc    papi/5.5.1    pmix/1.2.3    prun/1.2    singularity/2.4
   valgrind/3.13

module load openmpi/1.10.7 
module avail

----------------- /opt/ohpc/pub/moduledeps/gnu7-openmpi -----------------------------------
   petsc/3.7.6    phdf5/1.10.1    scalapack/2.0.2

----------------- /opt/ohpc/pub/moduledeps/gnu7 -------------------------------------------
   R/3.4.2    gsl/2.4    hdf5/1.10.1    metis/5.1.0    mpich/3.2    mvapich2/2.2    numpy/1.13.1
   ocr/1.0.1  openblas/0.2.20    openmpi/1.10.7 (L)    openmpi3/3.0.0    pdtoolkit/3.24    plasma/2.8.0
   scotch/6.0.4    superlu/5.2.1

----------------- /opt/ohpc/pub/modulefiles -----------------------------------------------
   EasyBuild/3.4.1    autotools    clustershell/1.8    cmake/3.9.2    gnu/5.4.0    gnu7/7.2.0 (L) 
   hwloc/1.11.8 llvm4/4.0.1    ohpc    papi/5.5.1    pmix/1.2.3    prun/1.2    singularity/2.4 
   valgrind/3.13.0

No module are loaded per default.

To check what loading a module will do use module show <module>:

# module show gnu7/7.2.0
-------------------------------------------------------------------------------------------
   /opt/ohpc/pub/modulefiles/gnu7/7.2.0:
-------------------------------------------------------------------------------------------
whatis("Name: GNU Compiler Collection ")
whatis("Version: 7.2.0 ")
whatis("Category: compiler, runtime support ")
whatis("Description: GNU Compiler Family (C/C++/Fortran for x86_64) ")
whatis("URL: http://gcc.gnu.org/ ")
prepend_path("PATH","/opt/ohpc/pub/compiler/gcc/7.2.0/bin")
prepend_path("MANPATH","/opt/ohpc/pub/compiler/gcc/7.2.0/share/man")
prepend_path("INCLUDE","/opt/ohpc/pub/compiler/gcc/7.2.0/include")
prepend_path("LD_LIBRARY_PATH","/opt/ohpc/pub/compiler/gcc/7.2.0/lib64")
prepend_path("MODULEPATH","/opt/ohpc/pub/moduledeps/gnu7")
family("compiler")
help([[ 
This module loads the GNU compiler collection
 
See the man pages for gcc, g++, and gfortran for detailed information
on available compiler options and command-line syntax.
 

Version 7.2.0

]])

• setup your own modules directory and a directory for your own R modulefile

mkdir ${HOME}/mymodules
mkdir ${HOME}/mymodules/R

• prepend own module directory to MODULEPATH:

export MODULEPATH=${HOME}/mymodules:$MODULEPATH

• create a module file: ${HOME}/mymodules/R/3.4.2:

#%Module
set inst_base $env(HOME)/mysoftware/R/3.4.2

module-whatis "This is my own customized R installation"

prepend-path PATH $inst_base/bin
prepend-path LD_LIBRARY_PATH $inst_base/lib
append-path CPATH $inst_base/include
setenv MYVAR test1234

• check how a system module is loaded

module purge
module avail
module load R/3.4.2
module show R/3.4.2
module whatis R/3.4.2

• create your own module environment as described on previous slide
• now load your own R module (called my_R here) and check if the correct module has been loaded

module avail
module rm R/3.4.2
module load my_R/3.4.2
module show my_R/3.4.2
module whatis my_R/3.4.2

• and finally try all module commands listed on slide 4.

A simple hello world example can be found in /opt/ohpc/pub/examples/slurm/mul/mpi/hello.c.
The modules gnu7/7.2.0 and openmpi/1.10.7 should be loaded for this example:

mpicc -show
gcc -I/opt/ohpc/pub/mpi/openmpi-gnu7/1.10.7/include -pthread -Wl,-rpath
   -Wl,/opt/ohpc/pub/mpi/openmpi-gnu7/1.10.7/lib
   -Wl,--enable-new-dtags -L/opt/ohpc/pub/mpi/openmpi-gnu7/1.10.7/lib -lmpi

Compile the code:

mpicc hello.c -o hello

Run the executeable:

mpirun -np 2 ./hello
 Hello, world (2 procs total)
    --> Process #   0 of   2 is alive.
    --> Process #   1 of   2 is alive.

• MPI is configured in such a way, that it will communicate via the infiniband network.
• Therefore use MPI on nodes which are connected via Infiniband.
• Make use of (OpenMP) threads when jobs run node local.
• Run codes in hybrid mode when possible.

• create a directory for the source code and copy the example C code:

mkdir helloworld
cd helloworld/
cp /opt/ohpc/pub/examples/slurm/mul/mpi/hello.c .

• load the required modules:

module purge
module load gnu7/7.2.0 openmpi/1.10.7

• compile and run the program:

mpicc hello.c -o hello
mpirun -np 2 ./hello