User Tools

Site Tools


doku:forge

VSC-3: forge = map + ddt

Synopsis:

map and ddt are Allinea's http://www.allinea.com/products advanced tools for performance analysis and licenses have been acquired for VSC-3 (up to 512 parallel tasks).

Usage of map:

Profiling may be split into two steps, where the initial task is to create a *.map file from within a regular job script submitted to SLURM. In a subsequent step this *.map file can then be analyzed within an interactive session on the login node. Suppose we had previously prepared an application for profiling, for instance via mpicc -g -O3 ./my_prog.c then we could call for a corresponding profile with the following submit script,

 #!/bin/bash
 #
 #SBATCH -J map       
 #SBATCH -N 4
 #SBATCH -L allinea@vsc
 #SBATCH --ntasks-per-node=16
 #SBATCH --ntasks-per-core=1
 
 module purge
 module load allinea/6.0.4_FORGE  intel-mpi/5  intel/14.0.2
 
 map --profile mpirun -np $SLURM_NTASKS ./a.out
 

which generates a *.map file that may then be analyzed with the help of the gui, ie

 ssh vsc3.vsc.ac.at -l my_uid -X
 cd wherever/the/map/file/may/be
 
 module purge
 module load allinea/6.0.4_FORGE
      
 map ./my_app.map

Usage of ddt:

Debugging with ddt is currently limited to the Remote Launch option. There are two ways to launch this type of ddt-sessions:

1.) ddt (semi-interactive via sbatch):

The following steps need to be carried out:

 ssh vsc3.vsc.ac.at -l my_uid -X
 my_uid@l33$  module load allinea/6.0.4_FORGE 
 my_uid@l33$  ddt &
              ... select 'Remote Launch - Configure'
              ... click  'Add' ( or select a previous session and modify it by clicking 'Edit' )  
              ... give it some name, which may be re-used/edited in future
              ... set my_uid@vsc3.vsc.ac.at as 'Host Name' 
              ... set 'Remote Installation Directory' to /opt/sw/x86_64/glibc-2.12/ivybridge-ep/allinea/6.0.4
              ... check it with 'Test Remote Launch'     ( will ask for a Password/OTP then monitor successful testing )
              ... click OK to close the dialogue
              ... click Close to exit from the Configure menu
              ... next really select 'Remote Launch' by clicking the name given above   ( will again ask for a Password/OTP, then the licence label should come up ok in the lower left corner and the connecting client should appear in the lower right corner )
              
 ssh vsc3.vsc.ac.at -l my_uid   ( a second terminal will be needed to actually start the debug session )
 my_uid@l35$  cd wherever/my/app/may/be
 my_uid@l35$  module load intel/14.0.2 intel-mpi/5    ( or whatever else suite of MPI )
 my_uid@l35$  mpiicc -g -O0 my_app.c
 my_uid@l35$  vi  ./run.ddt.slrm.scrpt
              ... insert the usual '#SBATCH ...' commands
              ... don't forget to include 'module load allinea/6.0.4_FORGE' and '#SBATCH -L allinea@vsc'
              ... the actual program execution should read, 'ddt --connect mpirun -np ...'
 my_uid@l35$  sbatch ./run.ddt.slrm.scrpt  ( in the other ddt-window sent into the background initially, a separate window pops up saying 'Reverse Connect Request' which needs to be accepted, then the usual ddt options will become available and the actual session may be launched by clicking 'Run' )

2.) ddt (fully interactive via salloc):

The following steps need to be carried out:

 ssh vsc3.vsc.ac.at -l my_uid -X
 my_uid@l33$  module load allinea/6.0.4_FORGE intel/14.0.2 intel-mpi/5
 my_uid@l33$  cd wherever/my/app/may/be
 my_uid@l33$  mpicc -g -O0 my_app.c
 my_uid@l33$  salloc -N 4
 my_uid@l33$  scontrol show hostnames $SLURM_NODELIST  > ./machines.txt
 my_uid@l33$  squeue -u my_uid   ( or just look into ./machines.txt to figure out which particular nodes have been assigned; let's assume we got n12-[045,051,068,071] )
 my_uid@l33$  ddt &
              ... select 'Remote Launch - Configure'
              ... click  'Add'   
              ... give it some name, which may be re-used/edited in future sessions
              ... set my_uid@n12-045 as 'Host Name' or any other node from the above list
              ... set 'Remote Installation Directory' to /opt/sw/x86_64/glibc-2.12/ivybridge-ep/allinea/6.0.4
              ... check it with 'Test Remote Launch'     ( should be ok )
              ... click OK to close the dialogue
              ... click Close to exit from the Configure menu
              ... next really select 'Remote Launch' by clicking the name given above   ( licence should be ok and the connecting client should appear in the lower right corner )
              
 ssh vsc3.vsc.ac.at -l my_uid   ( a second terminal will be needed to actually start the debug session )
 
 my_uid@l34$  ssh n12-045       ( log into the compute node selected and prepared for remote launch )
 my_uid@n12-045$  module load allinea/6.0.4_FORGE intel/14.0.2 intel-mpi/5
 my_uid@n12-045$  cd wherever/my/app/may/be
 my_uid@n12-045$  mpirun -np 16  -machinefile ./machines.txt  hostname   ( just a dummy check to see whether all is set up correctly )
 my_uid@n12-045$  ddt --connect mpirun -np 64  -machinefile ./machines.txt ./a.out -arg1 -arg2   ( another window will pop up prompting for a Reverse Connect request; accept it and the usual debug session will start )

Further Reading:

doku/forge.txt · Last modified: 2016/07/18 13:15 by sh