Differences

This shows you the differences between two versions of the page.

--- doku:spack-transition [2023/05/04 08:18] – [Current problems] katrin
+++ doku:spack-transition [2023/05/10 11:18] – spack search msiegel
@@ Line 1: / Line 1: @@
-====== SPACK - migration to setup without environment  ======
+====== SPACK - migration to setup without environment ======
-===== Problems of SPACK environments =====
+===== Motivation: problems of SPACK environments =====
 Having worked with //spack environments// for some time, we have encountered several severe issues which have convinced us that we need to find a more practical way of maintaining software packages on VSC.
@@ Line 12: / Line 12: @@
   * unreliable concretisation procedure (discrepancies between what is shown in ''spack spec -I ..'' and what is actually installed with ''spack install ...'').
-===== New approach without environments =====
+==== New approach without environments ====
-There are three separate spack installation trees corresponding to the CPU/GPU architectures on VSC:
+There are now three separate spack installation trees corresponding to the CPU/GPU architectures on VSC:
   * skylake - Intel CPUs; works on Intel Skylake and Cascadelake CPUs
@@ Line 26: / Line 26: @@
 </code>
-<box 80% round blue|Info> Please note that the prefix (e.g. 'zen') does **not** mean that a python virtual environment is loaded.</box>
+<box 80% round blue|Info> Please note that the prefix (e.g. ''zen'') does **not** mean that a python virtual environment is loaded.</box>
 The installation trees can be found in:
@@ Line 38: / Line 38: @@
 Only the software packages and modules of the currently active tree (denoted by the prefix) will be searched by ''spack find/load'' and ''module avail/load'' commands.
-It is easily possible to switch between installation trees with the short commands:
+It is easily possible to switch between installation trees with the short commands ''cuz'',''zen'', and ''sky'' defined as aliases:
 <code>
@@ Line 46: / Line 46: @@
 </code>
-This will be required e.g. when software intended to run on GPU nodes needs to be compiled on a login node of VSC-5.
+Packages depending on GPU, like ''cuda'' are only installed in the ''cuda-zen'' tree. If you want to compile software intended to run on GPU nodes, you need to:
+  - login on a VSC-5 login node (or a VSC-5 compute node).
+  - type ''cuz'' switch to ''cuda-zen''.
+  - compile your code.
 <code>
 zen [userl51 ~]$ cuz
-cuz [userl51 ~]$
+cuda-zen [userl51 ~]$
 ... build your software ...
 </code>
-The command ''spackup'' will
+The commands ''cuz''/''zen''/''sky'' ultimately call the shell function ''spackup <arch>'' which will:
   * set PATH
   * set MODULEPATH
@@ Line 60: / Line 63: @@
   * source /path/to/spack/instance/share/spack/setup-env.sh
+You can view the shell function ''spackup'' with ''type spackup'', or take a look at the whole script at ''/etc/profile.d/spack.sh''.
 ----
-===== Migration script - search.py =====
+===== Migration script - "spack search" =====
-If you want to find a list of packages or modules which correspond to the software you have been using until now, you may use the following script:
+If you need to find a list of packages or modules which correspond to the software you have been using until now, you can use the shell function ''spack search'' to do that:
 <code>
-zen [root@l51 ~] python3 /opt/sw/spack-scripts/search/search.py <hash>
+zen [user@l51 ~] spack search <hash>
 </code>
 where ''<hash>'' is the 7 character hash of the package that you see at the end of the module name or in the output of ''spack find -l <package>''.
 ==== Usage Example ====
-Let's assume you have used this netcdf-c module:
+Let's assume you have used this ''netcdf-c'' module:
 <code>
@@ Line 87: / Line 92: @@
 </code>
-You can use the migration script to find netcdf-c installations with identical build options:
+You can use the migration script to find ''netcdf-c'' installations with identical build options:
 <code>
-zen [root@l51 ~]$ module load python/3.9.15-gcc-12.2.0-my6jxu2
+zen [user@l51 ~]$ spack search jsfjwaz
-zen [root@l51 ~]$ python3 /opt/sw/spack-scripts/search/search.py jsfjwaz
+==> The hash 'jsfjwaz7qp52fjxfeg6mbhtt2lj3l573' refers to 'netcdf-c' from 'vsc5' with parameters:
-==> The package hash jsfjwaz refers to netcdf-c and belongs to vsc5 as:
      netcdf-c ~dap~fsync~hdf4~jna+mpi~parallel-netcdf+pic+shared
-==> Searching similar netcdf-c modules at zen...
+==> Searching similar 'netcdf-c' modules in installation 'skylake' ...
----/gpfs/opt/sw/zen/spack-0.19.0/share/spack/modules/linux-almalinux8-zen3---
+-- /opt/sw/skylake/spack-0.19.0/share/spack/modules/linux-almalinux8-skylake_avx512 --
-netcdf-c/4.9.0-aocc-4.0.0-6fetjps
+netcdf-c/4.9.0-gcc-12.2.0-xck6m4e
-netcdf-c/4.9.0-aocc-4.0.0-vyf5okn
+netcdf-c/4.9.0-gcc-12.2.0-vcjclck
-netcdf-c/4.9.0-gcc-12.2.0-usqc3ay
+netcdf-c/4.9.0-intel-2021.7.1-u6wt7yr
-netcdf-c/4.9.0-gcc-12.2.0-oljzgqt
+netcdf-c/4.9.0-intel-2021.7.1-k2p5vx2
-netcdf-c/4.9.0-gcc-12.2.0-hbo7ve6
-netcdf-c/4.9.0-gcc-12.2.0-62buxj4
-netcdf-c/4.9.0-gcc-12.2.0-awvdfta
-netcdf-c/4.9.0-gcc-12.2.0-oppx7bm
----/gpfs/opt/sw/zen/spack-0.19.0/share/spack/modules/linux-almalinux8-zen---
+==> Load any one of these packages with 'module load mypackage' e.g.:
-netcdf-c/4.9.0-gcc-8.5.0-7qwm3uz
+     module load netcdf-c/4.9.0-intel-2021.7.1-k2p5vx2
-==> Load any one of these packages with 'module load mypackage', e.g.:
+==> Get detailed package info with 'spack find -lvd mypackage' e.g.:
-module load netcdf-c/4.9.0-gcc-8.5.0-7qwm3uz
+     spack find -lvd /k2p5vx
-==> Get some additional info with 'spack find -lvd mypackage', e.g.:
-spack find -lvd /7qwm3u
 </code>
-You can also switch to another spack tree and search for packages there, e.g. cuda-zen:
+To search for packages in a different spack tree you can just change to that tree with ''cuz''/''sky''/''zen'', and then run ''spack search'' there.
+For example to search in the ''cuda-zen'' spack tree:
 <code>
-zen [root@l51 search]$ cuz
+zen [user@l51 search]$ cuz
-cuda-zen [root@l51 search]$ python3 ./search.py jsfjwaz
+cuda-zen [user@l51 search]$ spacksearch jsfjwaz
 ==> The package hash jsfjwaz refers to netcdf-c and belongs to vsc5 as:
@@ Line 135: / Line 133: @@
 netcdf-c/4.9.0-gcc-9.5.0-o5eb5rf
 netcdf-c/4.9.0-gcc-9.5.0-upkxxip
-==> Load any one of these packages with 'module load mypackage', e.g.:
-module load netcdf-c/4.9.0-gcc-9.5.0-upkxxip
-==> Get some additional info with 'spack find -lvd mypackage', e.g.:
-spack find -lvd /upkxxi
 </code>
+----
-===== Continue working with old modules (from environments) =====
+==== Deprecated: Continue working with old modules (from environments) ====
+<box 80% round red|Warning>This approach is not recommended and we will no longer maintain the spack environments or install new packages there</box>
 You may continue to use modules from the spack environments skylake and zen3 by adjusting the ''MODULEPATH'' variable:
-For ''zen3'':
+=== On zen3 ===
 <code>
 export MODULEPATH=/opt/sw/vsc4/VSC/Modules/TUWien:/opt/sw/vsc4/VSC/Modules/Intel/oneAPI:/opt/sw/vsc4/VSC/Modules/Parallel-Environment:/opt/sw/vsc4/VSC/Modules/Libraries:/opt/sw/vsc4/VSC/Modules/Compiler:/opt/sw/vsc4/VSC/Modules/Debugging-and-Profiling:/opt/sw/vsc4/VSC/Modules/Applications:/opt/sw/vsc4/VSC/Modules/p71545::/opt/sw/spack-0.17.1/var/spack/environments/zen3/modules/linux-almalinux8-zen:/opt/sw/spack-0.17.1/var/spack/environments/zen3/modules/linux-almalinux8-zen2:/opt/sw/spack-0.17.1/var/spack/environments/zen3/modules/linux-almalinux8-zen3
 </code>
-For ''skylake'':
+=== On skylake ===
 <code>
 export MODULEPATH=/opt/sw/vsc4/VSC/Modules/TUWien:/opt/sw/vsc4/VSC/Modules/Intel/oneAPI:/opt/sw/vsc4/VSC/Modules/Parallel-Environment:/opt/sw/vsc4/VSC/Modules/Libraries:/opt/sw/vsc4/VSC/Modules/Compiler:/opt/sw/vsc4/VSC/Modules/Debugging-and-Profiling:/opt/sw/vsc4/VSC/Modules/Applications:/opt/sw/vsc4/VSC/Modules/p71545:/opt/sw/vsc4/VSC/Modules/p71782::/opt/sw/spack-0.19.0/var/spack/environments/skylake/modules/linux-almalinux8-x86_64:/opt/sw/spack-0.19.0/var/spack/environments/skylake/modules/linux-almalinux8-skylake
 </code>
+----
+==== Deprecated: Continue working with old spack environments ====
 <box 80% round red|Warning>This approach is not recommended and we will no longer maintain the spack environments or install new packages there</box>
+You may also continue to work with the spack environments. If you wish to do so you need the following commands:
+=== On zen3 ===
+<code>
+export SPACK_ROOT=/opt/sw/spack-0.17.1
+source /opt/sw/spack-0.17.1/share/spack/setup-env.sh
+spacktivate zen3
+</code>
+=== On skylake ===
+<code>
+export SPACK_ROOT=/opt/sw/spack-0.19.0
+source /opt/sw/spack-0.19.0/share/spack/setup-env.sh
+spacktivate skylake
+</code>
 ----
@@ Line 166: / Line 186: @@
 <code>
-zen [root@l51 ~]$ module show py-numpy/1.23.4-gcc-12.2.0-xbac5zw |grep prereq
+zen [user@l51 ~]$ module show py-numpy/1.23.4-gcc-12.2.0-xbac5zw |grep prereq
 prereq          openblas/0.3.21-gcc-12.2.0-gcn6jxp
 prereq          py-setuptools/59.4.0-gcc-12.2.0-qphisr6
@@ Line 175: / Line 195: @@
 <code>
-zen [root@l51 ~]$ module load --auto py-numpy/1.23.4-gcc-12.2.0-xbac5zw
+zen [user@l51 ~]$ module load --auto py-numpy/1.23.4-gcc-12.2.0-xbac5zw
 Loading py-numpy/1.23.4-gcc-12.2.0-xbac5zw
   Loading requirement: openblas/0.3.21-gcc-12.2.0-gcn6jxp python/3.9.15-gcc-12.2.0-my6jxu2 py-setuptools/59.4.0-gcc-12.2.0-qphisr6
@@ Line 193: / Line 213: @@
 ----