This version is outdated by a newer approved version.This version (2014/06/18 08:24) is a draft.
Approvals: 0/1
Approvals: 0/1
This is an old revision of the document!
Recommendations
General recommendations
To make sure that the MPI communication happens via the infiniband fabric, please use the following settings in your job-script and/or in your .bashrc file:
export I_MPI_DAT_LIBRARY=/usr/lib64/libdat2.so.2 export OMP_NUM_THREADS=1 export I_MPI_FABRICS=shm:dapl export I_MPI_FALLBACK=0 export I_MPI_CPUINFO=proc export I_MPI_PIN_PROCESSOR_LIST=1,14,9,6,5,10,13,2,3,12,11,4,7,8,15,0 export I_MPI_JOB_FAST_STARTUP=0
Recommendations for various codes
Recommendations for advanced users
- fft libraries
- large jobs with more than 1024 cores
- memory intensive jobs requiring more than 2 GB per core
- ScaLAPACK compile options
- user defined prolog and epilog scripts