User Tools

Site Tools


doku:forge

VSC-3: forge = map + ddt

Synopsis:

map and ddt are Allinea's http://www.allinea.com/products advanced tools for performance analysis and licenses have been acquired for VSC-3 (up to 512 parallel tasks).

Usage of map:

Profiling may be split into two steps, where the initial task is to create a *.map file from within a regular job script submitted to SLURM. In a subsequent step this *.map file can then be analyzed within an interactive session on the login node. Suppose we had previously prepared an application for profiling, for instance via mpicc -g -O3 ./my_prog.c then we could call for a corresponding profile with the following submit script,

 #!/bin/bash
 #
 #SBATCH -J map       
 #SBATCH -N 4
 #SBATCH -L allinea@vsc
 #SBATCH --ntasks-per-node=16
 #SBATCH --ntasks-per-core=1
 
 module purge
 module load  intel/16.0.4  intel-mpi/5.1.3  allinea/7.0.1_FORGE
 
 map --profile mpirun -np $SLURM_NTASKS ./a.out
 

which generates a *.map file that may then be analyzed with the help of the gui, ie

 ssh vsc3.vsc.ac.at -l my_uid -X
 cd wherever/the/map/file/may/be
 
 module purge
 module load allinea/7.0.1_FORGE
      
 map ./my_app.map

Usage of ddt:

Debugging with ddt is currently limited to the Remote Launch option. There are two ways to launch this type of ddt-sessions:

1.) ddt (semi-interactive via sbatch):

The following steps need to be carried out:

 ssh vsc3.vsc.ac.at -l my_uid -X
 my_uid@l33$  module load allinea/7.0.1_FORGE 
 my_uid@l33$  rm -rf ~/.allinea/  ( to get rid of obsolete configurations from previous sessions )
 my_uid@l33$  ddt &
              ... select 'Remote Launch - Configure'
              ... click  'Add' 
              ... set my_uid@l33.vsc.ac.at as 'Host Name' depending on which l3[1-5] node we currently are
              ... set 'Remote Installation Directory' to /opt/sw/x86_64/glibc-2.17/ivybridge-ep/allinea/7.0.1_FORGE
              ... check it with 'Test Remote Launch'     ( will ask for a Password/OTP then monitor successful testing )
              ... click OK twice to close the dialogues
              ... click Close to exit from the Configure menu
              ... next really select 'Remote Launch' by clicking the name tag that was auto-assigned   ( will again ask for a Password/OTP, then the licence label should come up ok in the lower left corner and the connecting client should appear in the lower right corner )
              
 ssh vsc3.vsc.ac.at -l my_uid   ( a second terminal will be needed to actually start the debug session )
 my_uid@l35$  cd wherever/my/app/may/be
 my_uid@l35$  module load intel/16.0.4  intel-mpi/5.1.3   ( or whatever else suite of MPI )
 my_uid@l35$  mpiicc -g -O0 my_app.c
 my_uid@l35$  vi  ./run.ddt.slrm.scrpt
              ... insert the usual '#SBATCH ...' commands
              ... don't forget to include 'module load allinea/7.0.1_FORGE' and '#SBATCH -L allinea@vsc'
              ... the actual program execution should be prefixed with 'ddt --connect mpirun -np ...'
 my_uid@l35$  sbatch ./run.ddt.slrm.scrpt  ( in the other ddt-window sent into the background initially, a separate window pops up saying 'Reverse Connect Request' which needs to be accepted, then the usual ddt options will become available and the actual session may be launched by clicking 'Run' )

2.) ddt (fully interactive via salloc):

The following steps need to be carried out:

 ssh vsc3.vsc.ac.at -l my_uid -X
 my_uid@l33$  module load  intel/16.0.4  intel-mpi/5.1.3  allinea/7.0.1_FORGE   ( or whatever else suite of MPI )
 my_uid@l33$  cd wherever/my/app/may/be
 my_uid@l33$  mpicc -g -O0 my_app.c
 my_uid@l33$  salloc -N 4 -L allinea@vsc
 my_uid@l33$  scontrol show hostnames $SLURM_NODELIST  > ./machines.txt
              ... let's assume we got n12-[045,051,068,071] which should now be listed inside file 'machines.txt' 
 my_uid@l33$  rm -rf ~/.allinea/   ( to get rid of obsolete configurations from previous sessions )
 my_uid@l33$  srun -n 4 hostname   ( this is important ! it looks like a redundant command but will actually fix a lot of the prerequisites usually taken care of in the SLURM prologue of regular submit scripts, one of them being provisioning of required licenses )
 my_uid@l33$  ddt &
              ... select 'Remote Launch - Configure'
              ... click  'Add'   
              ... set my_uid@n12-045 as 'Host Name' or any other node from the above list
              ... set 'Remote Installation Directory' to /opt/sw/x86_64/glibc-2.17/ivybridge-ep/allinea/7.0.1_FORGE
              ... check it with 'Test Remote Launch'     ( should be ok )
              ... click OK twice to close the dialogues
              ... click Close to exit from the Configure menu
              ... next really select 'Remote Launch' by clicking the name tag that was auto-assigned above   ( licence label should be ok in the lower left corner and the connecting client should appear in the lower right corner )
              
 ssh vsc3.vsc.ac.at -l my_uid   ( a second terminal will be needed to actually start the debug session )   
 my_uid@l34$  ssh n12-045       ( log into that compute node that was selected/prepared above for remote launch )
 my_uid@n12-045$  module load  intel/16.0.4  intel-mpi/5.1.3  allinea/7.0.1_FORGE
 my_uid@n12-045$  cd wherever/my/app/may/be
 my_uid@n12-045$  mpirun -np 16  -machinefile ./machines.txt  hostname   ( just a dummy check to see whether all is set up correctly )
 my_uid@n12-045$  ddt --connect mpirun -np 64  -machinefile ./machines.txt ./a.out -arg1 -arg2   ( in the initial ddt-window a dialogue will pop up prompting for a Reverse Connection request; accept it and click Run and the usual debug session will start )

Further Reading:

doku/forge.txt · Last modified: 2017/05/04 15:59 by sh