Both sides previous revision Previous revision Next revision | Previous revision |
doku:spack-transition [2023/05/04 08:44] – [Usage Example] katrin | doku:spack-transition [2023/05/17 14:48] (current) – msiegel |
---|
====== SPACK - migration to setup without environment ====== | ====== SPACK - migration to setup without environment ====== |
| |
===== Problems of SPACK environments ===== | ===== Motivation: problems of SPACK environments ===== |
| |
Having worked with //spack environments// for some time, we have encountered several severe issues which have convinced us that we need to find a more practical way of maintaining software packages on VSC. | Having worked with //spack environments// for some time, we have encountered several severe issues which have convinced us that we need to find a more practical way of maintaining software packages on VSC. |
* unreliable concretisation procedure (discrepancies between what is shown in ''spack spec -I ..'' and what is actually installed with ''spack install ...''). | * unreliable concretisation procedure (discrepancies between what is shown in ''spack spec -I ..'' and what is actually installed with ''spack install ...''). |
| |
===== New approach without environments ===== | ==== New approach without environments ==== |
| |
There are three separate spack installation trees corresponding to the CPU/GPU architectures on VSC: | There are now three separate spack installation trees corresponding to the CPU/GPU architectures on VSC: |
| |
* skylake - Intel CPUs; works on Intel Skylake and Cascadelake CPUs | * skylake - Intel CPUs; works on Intel Skylake and Cascadelake CPUs |
</code> | </code> |
| |
<box 80% round blue|Info> Please note that the prefix (e.g. 'zen') does **not** mean that a python virtual environment is loaded.</box> | <box 80% round blue|Info> Please note that the prefix (e.g. ''zen'') does **not** mean that a python virtual environment is loaded.</box> |
| |
The installation trees can be found in: | The installation trees can be found in: |
Only the software packages and modules of the currently active tree (denoted by the prefix) will be searched by ''spack find/load'' and ''module avail/load'' commands. | Only the software packages and modules of the currently active tree (denoted by the prefix) will be searched by ''spack find/load'' and ''module avail/load'' commands. |
| |
It is easily possible to switch between installation trees with the short commands: | It is easily possible to switch between installation trees with the short commands ''cuz'',''zen'', and ''sky'' defined as aliases: |
| |
<code> | <code> |
</code> | </code> |
| |
This will be required e.g. when software intended to run on GPU nodes needs to be compiled on a login node of VSC-5. | Packages depending on GPU, like ''cuda'' are only installed in the ''cuda-zen'' tree. If you want to compile software intended to run on GPU nodes, you need to: |
| - login on a VSC-5 login node (or a VSC-5 compute node). |
| - type ''cuz'' switch to ''cuda-zen''. |
| - compile your code. |
| |
<code> | <code> |
zen [userl51 ~]$ cuz | zen [userl51 ~]$ cuz |
cuz [userl51 ~]$ | cuda-zen [userl51 ~]$ |
... build your software ... | ... build your software ... |
</code> | </code> |
| |
The command ''spackup'' will | The commands ''cuz''/''zen''/''sky'' ultimately call the shell function ''spackup <arch>'' which will: |
* set PATH | * set PATH |
* set MODULEPATH | * set MODULEPATH |
* source /path/to/spack/instance/share/spack/setup-env.sh | * source /path/to/spack/instance/share/spack/setup-env.sh |
| |
| You can view the shell function ''spackup'' with ''type spackup'', or take a look at the whole script at ''/etc/profile.d/spack.sh''. |
| |
---- | ---- |
| |
===== Migration script - "spacksearch" ===== | ===== Migration script - "spack search" ===== |
| |
If you need to find a list of packages or modules which correspond to the software you have been using until now, you can use the alias **spacksearch** to do that | If you need to find a list of packages or modules which correspond to the software you have been using until now, you can use the shell function ''spack search'' to do that: |
| |
<code> | <code> |
alias spacksearch='python3 /opt/sw/spack-scripts/search/search.py' | zen [user@l51 ~] spack search <hash> |
</code> | </code> |
| |
The script can be used like in the following example: | where ''<hash>'' is the 7 character hash of the package that you see at the end of the module name or in the output of ''spack find -l <package>''. |
| |
<code> | |
zen [user@l51 ~] spacksearch <hash> | |
</code> | |
| |
where ''<hash>'' is the 7 character hash of the package that you see at the end of the module name or in the output of ''spack find -l <package>'' | |
| |
==== Usage Example ==== | ==== Usage example ==== |
| |
Let's assume you have used this netcdf-c module: | Let's assume you have used this ''netcdf-c'' module: |
| |
<code> | <code> |
</code> | </code> |
| |
You can use the migration script to find netcdf-c installations with identical build options: | You can use the migration script to find ''netcdf-c'' installations with identical build options: |
| |
<code> | <code> |
zen [user@l51 ~]$ spacksearch jsfjwaz | zen [user@l51 ~]$ spack search jsfjwaz |
| |
==> The package hash jsfjwaz refers to netcdf-c and belongs to vsc5 as: | ==> The hash 'jsfjwaz7qp52fjxfeg6mbhtt2lj3l573' refers to 'netcdf-c' from 'vsc5' with parameters: |
netcdf-c ~dap~fsync~hdf4~jna+mpi~parallel-netcdf+pic+shared | netcdf-c ~dap~fsync~hdf4~jna+mpi~parallel-netcdf+pic+shared |
| |
==> Searching similar netcdf-c modules at zen... | ==> Searching similar 'netcdf-c' modules in installation 'skylake' ... |
| |
---/gpfs/opt/sw/zen/spack-0.19.0/share/spack/modules/linux-almalinux8-zen3--- | -- /opt/sw/skylake/spack-0.19.0/share/spack/modules/linux-almalinux8-skylake_avx512 -- |
netcdf-c/4.9.0-aocc-4.0.0-6fetjps | netcdf-c/4.9.0-gcc-12.2.0-xck6m4e |
netcdf-c/4.9.0-aocc-4.0.0-vyf5okn | netcdf-c/4.9.0-gcc-12.2.0-vcjclck |
netcdf-c/4.9.0-gcc-12.2.0-usqc3ay | netcdf-c/4.9.0-intel-2021.7.1-u6wt7yr |
netcdf-c/4.9.0-gcc-12.2.0-oljzgqt | netcdf-c/4.9.0-intel-2021.7.1-k2p5vx2 |
netcdf-c/4.9.0-gcc-12.2.0-hbo7ve6 | |
netcdf-c/4.9.0-gcc-12.2.0-62buxj4 | |
netcdf-c/4.9.0-gcc-12.2.0-awvdfta | |
netcdf-c/4.9.0-gcc-12.2.0-oppx7bm | |
| |
---/gpfs/opt/sw/zen/spack-0.19.0/share/spack/modules/linux-almalinux8-zen--- | ==> Load any one of these packages with 'module load mypackage' e.g.: |
netcdf-c/4.9.0-gcc-8.5.0-7qwm3uz | module load netcdf-c/4.9.0-intel-2021.7.1-k2p5vx2 |
| |
==> Load any one of these packages with 'module load mypackage', e.g.: | ==> Get detailed package info with 'spack find -lvd mypackage' e.g.: |
module load netcdf-c/4.9.0-gcc-8.5.0-7qwm3uz | spack find -lvd /k2p5vx |
| |
==> Get some additional info with 'spack find -lvd mypackage', e.g.: | |
spack find -lvd /7qwm3u | |
</code> | </code> |
| |
To search for packages in a different spack tree you can just change to that tree and then run **spacksearch** there. | To search for packages in a different spack tree you can just change to that tree with ''cuz''/''sky''/''zen'', and then run ''spack search'' there. |
| |
For example to search in the 'cuda-zen' spack tree: | For example to search in the ''cuda-zen'' spack tree: |
| |
<code> | <code> |
netcdf-c/4.9.0-gcc-9.5.0-o5eb5rf | netcdf-c/4.9.0-gcc-9.5.0-o5eb5rf |
netcdf-c/4.9.0-gcc-9.5.0-upkxxip | netcdf-c/4.9.0-gcc-9.5.0-upkxxip |
| |
==> Load any one of these packages with 'module load mypackage', e.g.: | |
module load netcdf-c/4.9.0-gcc-9.5.0-upkxxip | |
| |
==> Get some additional info with 'spack find -lvd mypackage', e.g.: | |
spack find -lvd /upkxxi | |
</code> | </code> |
| |
| ---- |
| |
===== Continue working with old modules (from environments) ===== | ==== Deprecated: Continue working with old modules (from environments) ==== |
| |
| <box 80% round red|Warning>This approach is not recommended and we will no longer maintain the spack environments or install new packages there</box> |
| |
You may continue to use modules from the spack environments skylake and zen3 by adjusting the ''MODULEPATH'' variable: | You may continue to use modules from the spack environments skylake and zen3 by adjusting the ''MODULEPATH'' variable: |
| |
For ''zen3'': | === On zen3 === |
<code> | <code> |
export MODULEPATH=/opt/sw/vsc4/VSC/Modules/TUWien:/opt/sw/vsc4/VSC/Modules/Intel/oneAPI:/opt/sw/vsc4/VSC/Modules/Parallel-Environment:/opt/sw/vsc4/VSC/Modules/Libraries:/opt/sw/vsc4/VSC/Modules/Compiler:/opt/sw/vsc4/VSC/Modules/Debugging-and-Profiling:/opt/sw/vsc4/VSC/Modules/Applications:/opt/sw/vsc4/VSC/Modules/p71545::/opt/sw/spack-0.17.1/var/spack/environments/zen3/modules/linux-almalinux8-zen:/opt/sw/spack-0.17.1/var/spack/environments/zen3/modules/linux-almalinux8-zen2:/opt/sw/spack-0.17.1/var/spack/environments/zen3/modules/linux-almalinux8-zen3 | export MODULEPATH=/opt/sw/vsc4/VSC/Modules/TUWien:/opt/sw/vsc4/VSC/Modules/Intel/oneAPI:/opt/sw/vsc4/VSC/Modules/Parallel-Environment:/opt/sw/vsc4/VSC/Modules/Libraries:/opt/sw/vsc4/VSC/Modules/Compiler:/opt/sw/vsc4/VSC/Modules/Debugging-and-Profiling:/opt/sw/vsc4/VSC/Modules/Applications:/opt/sw/vsc4/VSC/Modules/p71545::/opt/sw/spack-0.17.1/var/spack/environments/zen3/modules/linux-almalinux8-zen:/opt/sw/spack-0.17.1/var/spack/environments/zen3/modules/linux-almalinux8-zen2:/opt/sw/spack-0.17.1/var/spack/environments/zen3/modules/linux-almalinux8-zen3 |
</code> | </code> |
| |
For ''skylake'': | === On skylake === |
<code> | <code> |
export MODULEPATH=/opt/sw/vsc4/VSC/Modules/TUWien:/opt/sw/vsc4/VSC/Modules/Intel/oneAPI:/opt/sw/vsc4/VSC/Modules/Parallel-Environment:/opt/sw/vsc4/VSC/Modules/Libraries:/opt/sw/vsc4/VSC/Modules/Compiler:/opt/sw/vsc4/VSC/Modules/Debugging-and-Profiling:/opt/sw/vsc4/VSC/Modules/Applications:/opt/sw/vsc4/VSC/Modules/p71545:/opt/sw/vsc4/VSC/Modules/p71782::/opt/sw/spack-0.19.0/var/spack/environments/skylake/modules/linux-almalinux8-x86_64:/opt/sw/spack-0.19.0/var/spack/environments/skylake/modules/linux-almalinux8-skylake | export MODULEPATH=/opt/sw/vsc4/VSC/Modules/TUWien:/opt/sw/vsc4/VSC/Modules/Intel/oneAPI:/opt/sw/vsc4/VSC/Modules/Parallel-Environment:/opt/sw/vsc4/VSC/Modules/Libraries:/opt/sw/vsc4/VSC/Modules/Compiler:/opt/sw/vsc4/VSC/Modules/Debugging-and-Profiling:/opt/sw/vsc4/VSC/Modules/Applications:/opt/sw/vsc4/VSC/Modules/p71545:/opt/sw/vsc4/VSC/Modules/p71782::/opt/sw/spack-0.19.0/var/spack/environments/skylake/modules/linux-almalinux8-x86_64:/opt/sw/spack-0.19.0/var/spack/environments/skylake/modules/linux-almalinux8-skylake |
</code> | </code> |
| |
| ---- |
| |
| ==== Deprecated: Continue working with old spack environments ==== |
| |
<box 80% round red|Warning>This approach is not recommended and we will no longer maintain the spack environments or install new packages there</box> | <box 80% round red|Warning>This approach is not recommended and we will no longer maintain the spack environments or install new packages there</box> |
| |
| You may also continue to work with the spack environments. If you wish to do so you need the following commands: |
| |
| === On zen3 === |
| |
| <code> |
| export SPACK_ROOT=/opt/sw/spack-0.17.1 |
| source /opt/sw/spack-0.17.1/share/spack/setup-env.sh |
| spacktivate zen3 |
| </code> |
| |
| === On skylake === |
| |
| <code> |
| export SPACK_ROOT=/opt/sw/spack-0.19.0 |
| source /opt/sw/spack-0.19.0/share/spack/setup-env.sh |
| spacktivate skylake |
| </code> |
| |
---- | ---- |
| |
| |
---- | ---- |
| |
| |