This version is outdated by a newer approved version.
This version (2014/06/18 08:23) is a draft.
Approvals: 0/1
This version (2014/06/18 08:23) is a draft.Approvals: 0/1
This is an old revision of the document!
General recommendations
To make sure that the MPI communication happens via the infiniband fabric, please use the following settings in your job-script and/or in your .bashrc file:
export I_MPI_DAT_LIBRARY=/usr/lib64/libdat2.so.2 export OMP_NUM_THREADS=1 export I_MPI_FABRICS=shm:dapl export I_MPI_FALLBACK=0 export I_MPI_CPUINFO=proc export I_MPI_PIN_PROCESSOR_LIST=1,14,9,6,5,10,13,2,3,12,11,4,7,8,15,0 export I_MPI_JOB_FAST_STARTUP=0
Recommendations for various codes
Recommendations for advanced users
- fft libraries
- large jobs with more than 1024 cores
- memory intensive jobs requiring more than 2 GB per core
- ScaLAPACK compile options
- user defined prolog and epilog scripts