Differences

This shows you the differences between two versions of the page.

--- doku:spack-transition [2023/05/05 11:42] – [Continue working with old modules (from environments)] jz
+++ doku:spack-transition [2023/05/17 14:48] (current) – msiegel
@@ Line 1: / Line 1: @@
-====== SPACK - migration to setup without environment  ======
+====== SPACK - migration to setup without environment ======
-===== Problems of SPACK environments =====
+===== Motivation: problems of SPACK environments =====
 Having worked with //spack environments// for some time, we have encountered several severe issues which have convinced us that we need to find a more practical way of maintaining software packages on VSC.
@@ Line 12: / Line 12: @@
   * unreliable concretisation procedure (discrepancies between what is shown in ''spack spec -I ..'' and what is actually installed with ''spack install ...'').
-===== New approach without environments =====
+==== New approach without environments ====
 There are now three separate spack installation trees corresponding to the CPU/GPU architectures on VSC:
@@ Line 26: / Line 26: @@
 </code>
-<box 80% round blue|Info> Please note that the prefix (e.g. 'zen') does **not** mean that a python virtual environment is loaded.</box>
+<box 80% round blue|Info> Please note that the prefix (e.g. ''zen'') does **not** mean that a python virtual environment is loaded.</box>
 The installation trees can be found in:
@@ Line 38: / Line 38: @@
 Only the software packages and modules of the currently active tree (denoted by the prefix) will be searched by ''spack find/load'' and ''module avail/load'' commands.
-It is easily possible to switch between installation trees with the short commands:
+It is easily possible to switch between installation trees with the short commands ''cuz'',''zen'', and ''sky'' defined as aliases:
 <code>
@@ Line 46: / Line 46: @@
 </code>
-This will be required e.g. when software intended to run on GPU nodes needs to be compiled on a login node of VSC-5.
+Packages depending on GPU, like ''cuda'' are only installed in the ''cuda-zen'' tree. If you want to compile software intended to run on GPU nodes, you need to:
+  - login on a VSC-5 login node (or a VSC-5 compute node).
+  - type ''cuz'' switch to ''cuda-zen''.
+  - compile your code.
 <code>
 zen [userl51 ~]$ cuz
-cuz [userl51 ~]$
+cuda-zen [userl51 ~]$
 ... build your software ...
 </code>
-The command ''spackup'' will
+The commands ''cuz''/''zen''/''sky'' ultimately call the shell function ''spackup <arch>'' which will:
   * set PATH
   * set MODULEPATH
@@ Line 60: / Line 63: @@
   * source /path/to/spack/instance/share/spack/setup-env.sh
+You can view the shell function ''spackup'' with ''type spackup'', or take a look at the whole script at ''/etc/profile.d/spack.sh''.
 ----
-===== Migration script - "spacksearch" =====
+===== Migration script - "spack search" =====
-==== Method 1 ====
+If you need to find a list of packages or modules which correspond to the software you have been using until now, you can use the shell function ''spack search'' to do that:
-If you need to find a list of packages or modules which correspond to the software you have been using until now, you can use the alias **spacksearch** to do that
 <code>
-alias spacksearch='/usr/bin/python3.9 /opt/sw/spack-scripts/search/search.py'
+zen [user@l51 ~] spack search <hash>
 </code>
-The script can be used like in the following example:
+where ''<hash>'' is the 7 character hash of the package that you see at the end of the module name or in the output of ''spack find -l <package>''.
-<code>
-zen [user@l51 ~] spacksearch <hash>
-</code>
-where ''<hash>'' is the 7 character hash of the package that you see at the end of the module name or in the output of ''spack find -l <package>''
-==== Usage Example ====
+==== Usage example ====
-Let's assume you have used this netcdf-c module:
+Let's assume you have used this ''netcdf-c'' module:
 <code>
@@ Line 95: / Line 92: @@
 </code>
-You can use the migration script to find netcdf-c installations with identical build options:
+You can use the migration script to find ''netcdf-c'' installations with identical build options:
 <code>
-zen [user@l51 ~]$ spacksearch jsfjwaz
+zen [user@l51 ~]$ spack search jsfjwaz
-==> The package hash jsfjwaz refers to netcdf-c and belongs to vsc5 as:
+==> The hash 'jsfjwaz7qp52fjxfeg6mbhtt2lj3l573' refers to 'netcdf-c' from 'vsc5' with parameters:
      netcdf-c ~dap~fsync~hdf4~jna+mpi~parallel-netcdf+pic+shared
-==> Searching similar netcdf-c modules at zen...
+==> Searching similar 'netcdf-c' modules in installation 'skylake' ...
----/gpfs/opt/sw/zen/spack-0.19.0/share/spack/modules/linux-almalinux8-zen3---
-netcdf-c/4.9.0-aocc-4.0.0-6fetjps
-netcdf-c/4.9.0-aocc-4.0.0-vyf5okn
-netcdf-c/4.9.0-gcc-12.2.0-usqc3ay
-netcdf-c/4.9.0-gcc-12.2.0-oljzgqt
-netcdf-c/4.9.0-gcc-12.2.0-hbo7ve6
-netcdf-c/4.9.0-gcc-12.2.0-62buxj4
-netcdf-c/4.9.0-gcc-12.2.0-awvdfta
-netcdf-c/4.9.0-gcc-12.2.0-oppx7bm
----/gpfs/opt/sw/zen/spack-0.19.0/share/spack/modules/linux-almalinux8-zen---
+-- /opt/sw/skylake/spack-0.19.0/share/spack/modules/linux-almalinux8-skylake_avx512 --
-netcdf-c/4.9.0-gcc-8.5.0-7qwm3uz
+netcdf-c/4.9.0-gcc-12.2.0-xck6m4e
+netcdf-c/4.9.0-gcc-12.2.0-vcjclck
+netcdf-c/4.9.0-intel-2021.7.1-u6wt7yr
+netcdf-c/4.9.0-intel-2021.7.1-k2p5vx2
-==> Load any one of these packages with 'module load mypackage', e.g.:
+==> Load any one of these packages with 'module load mypackage' e.g.:
-module load netcdf-c/4.9.0-gcc-8.5.0-7qwm3uz
+     module load netcdf-c/4.9.0-intel-2021.7.1-k2p5vx2
-==> Get some additional info with 'spack find -lvd mypackage', e.g.:
+==> Get detailed package info with 'spack find -lvd mypackage' e.g.:
-spack find -lvd /7qwm3u
+     spack find -lvd /k2p5vx
 </code>
-To search for packages in a different spack tree you can just change to that tree and then run **spacksearch** there.
+To search for packages in a different spack tree you can just change to that tree with ''cuz''/''sky''/''zen'', and then run ''spack search'' there.
-For example to search in the 'cuda-zen' spack tree:
+For example to search in the ''cuda-zen'' spack tree:
 <code>
@@ Line 143: / Line 133: @@
 netcdf-c/4.9.0-gcc-9.5.0-o5eb5rf
 netcdf-c/4.9.0-gcc-9.5.0-upkxxip
+</code>
-==> Load any one of these packages with 'module load mypackage', e.g.:
+----
-module load netcdf-c/4.9.0-gcc-9.5.0-upkxxip
-==> Get some additional info with 'spack find -lvd mypackage', e.g.:
+==== Deprecated: Continue working with old modules (from environments) ====
-spack find -lvd /upkxxi
-</code>
+<box 80% round red|Warning>This approach is not recommended and we will no longer maintain the spack environments or install new packages there</box>
-==== Method 2 ====
+You may continue to use modules from the spack environments skylake and zen3 by adjusting the ''MODULEPATH'' variable:
-Add this function to your ''.bashrc'' file:
+=== On zen3 ===
 <code>
-## spack command wrapper, so we can include our own `spack something`
+export MODULEPATH=/opt/sw/vsc4/VSC/Modules/TUWien:/opt/sw/vsc4/VSC/Modules/Intel/oneAPI:/opt/sw/vsc4/VSC/Modules/Parallel-Environment:/opt/sw/vsc4/VSC/Modules/Libraries:/opt/sw/vsc4/VSC/Modules/Compiler:/opt/sw/vsc4/VSC/Modules/Debugging-and-Profiling:/opt/sw/vsc4/VSC/Modules/Applications:/opt/sw/vsc4/VSC/Modules/p71545::/opt/sw/spack-0.17.1/var/spack/environments/zen3/modules/linux-almalinux8-zen:/opt/sw/spack-0.17.1/var/spack/environments/zen3/modules/linux-almalinux8-zen2:/opt/sw/spack-0.17.1/var/spack/environments/zen3/modules/linux-almalinux8-zen3
-## commands, like `spack search`:
-spack () {
-    case "$1" in
-        "search")
-            python3 /opt/sw/spack-scripts/search/search.py "$2"
-            ;;
-        *)
-            command spack "$@"
-    esac
-}
 </code>
-Usage example:
+=== On skylake ===
 <code>
-skylake [user@l41 ~]$ spack search asd
+export MODULEPATH=/opt/sw/vsc4/VSC/Modules/TUWien:/opt/sw/vsc4/VSC/Modules/Intel/oneAPI:/opt/sw/vsc4/VSC/Modules/Parallel-Environment:/opt/sw/vsc4/VSC/Modules/Libraries:/opt/sw/vsc4/VSC/Modules/Compiler:/opt/sw/vsc4/VSC/Modules/Debugging-and-Profiling:/opt/sw/vsc4/VSC/Modules/Applications:/opt/sw/vsc4/VSC/Modules/p71545:/opt/sw/vsc4/VSC/Modules/p71782::/opt/sw/spack-0.19.0/var/spack/environments/skylake/modules/linux-almalinux8-x86_64:/opt/sw/spack-0.19.0/var/spack/environments/skylake/modules/linux-almalinux8-skylake
-==> The package hash asd refers to python and belongs to vsc4 as:
-     python +bz2+ctypes+dbm~debug+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tix~tkinter~ucs4+uuid+zlib
-==> Searching similar python modules at skylake...
----/gpfs/opt/sw/skylake/spack-0.19.0/share/spack/modules/linux-almalinux8-skylake_avx512---
-python/3.10.8-gcc-9.5.0-qh22vnd
-python/3.10.8-gcc-12.2.0-apbi5uz
-python/3.10.8-intel-2021.7.1-p4x6jid
-==> Load any one of these packages with 'module load mypackage', e.g.:
-module load python/3.10.8-intel-2021.7.1-p4x6jid
-==> Get some additional info with 'spack find -lvd mypackage', e.g.:
-spack find -lvd /p4x6ji
 </code>
 ----
+==== Deprecated: Continue working with old spack environments ====
-==== Continue working with old modules (from environments) ====
-You may continue to use modules from the spack environments skylake and zen3 by adjusting the ''MODULEPATH'' variable:
-For ''zen3'':
-<code>
-export MODULEPATH=/opt/sw/vsc4/VSC/Modules/TUWien:/opt/sw/vsc4/VSC/Modules/Intel/oneAPI:/opt/sw/vsc4/VSC/Modules/Parallel-Environment:/opt/sw/vsc4/VSC/Modules/Libraries:/opt/sw/vsc4/VSC/Modules/Compiler:/opt/sw/vsc4/VSC/Modules/Debugging-and-Profiling:/opt/sw/vsc4/VSC/Modules/Applications:/opt/sw/vsc4/VSC/Modules/p71545::/opt/sw/spack-0.17.1/var/spack/environments/zen3/modules/linux-almalinux8-zen:/opt/sw/spack-0.17.1/var/spack/environments/zen3/modules/linux-almalinux8-zen2:/opt/sw/spack-0.17.1/var/spack/environments/zen3/modules/linux-almalinux8-zen3
-</code>
-For ''skylake'':
-<code>
-export MODULEPATH=/opt/sw/vsc4/VSC/Modules/TUWien:/opt/sw/vsc4/VSC/Modules/Intel/oneAPI:/opt/sw/vsc4/VSC/Modules/Parallel-Environment:/opt/sw/vsc4/VSC/Modules/Libraries:/opt/sw/vsc4/VSC/Modules/Compiler:/opt/sw/vsc4/VSC/Modules/Debugging-and-Profiling:/opt/sw/vsc4/VSC/Modules/Applications:/opt/sw/vsc4/VSC/Modules/p71545:/opt/sw/vsc4/VSC/Modules/p71782::/opt/sw/spack-0.19.0/var/spack/environments/skylake/modules/linux-almalinux8-x86_64:/opt/sw/spack-0.19.0/var/spack/environments/skylake/modules/linux-almalinux8-skylake
-</code>
 <box 80% round red|Warning>This approach is not recommended and we will no longer maintain the spack environments or install new packages there</box>
-==== Continue working with old spack environments ====
 You may also continue to work with the spack environments. If you wish to do so you need the following commands:
-== On zen3 ==
+=== On zen3 ===
 <code>
+export SPACK_ROOT=/opt/sw/spack-0.17.1
 source /opt/sw/spack-0.17.1/share/spack/setup-env.sh
 spacktivate zen3
 </code>
-== On skylake ==
+=== On skylake ===
 <code>
+export SPACK_ROOT=/opt/sw/spack-0.19.0
 source /opt/sw/spack-0.19.0/share/spack/setup-env.sh
 spacktivate skylake
 </code>
-<box 80% round red|Warning>This approach is not recommended and we will no longer maintain the spack environments or install new packages there</box>
 ----
@@ Line 266: / Line 213: @@
 ----