Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
doku:spack-transition [2023/05/05 11:38] – [Continue working with old modules (from environments)] jz | doku:spack-transition [2023/05/17 14:48] (current) – msiegel | ||
---|---|---|---|
Line 1: | Line 1: | ||
- | ====== SPACK - migration to setup without environment | + | ====== SPACK - migration to setup without environment ====== |
- | ===== Problems | + | ===== Motivation: problems |
Having worked with //spack environments// | Having worked with //spack environments// | ||
Line 12: | Line 12: | ||
* unreliable concretisation procedure (discrepancies between what is shown in '' | * unreliable concretisation procedure (discrepancies between what is shown in '' | ||
- | ===== New approach without environments | + | ==== New approach without environments ==== |
There are now three separate spack installation trees corresponding to the CPU/GPU architectures on VSC: | There are now three separate spack installation trees corresponding to the CPU/GPU architectures on VSC: | ||
Line 26: | Line 26: | ||
</ | </ | ||
- | <box 80% round blue|Info> | + | <box 80% round blue|Info> |
The installation trees can be found in: | The installation trees can be found in: | ||
Line 38: | Line 38: | ||
Only the software packages and modules of the currently active tree (denoted by the prefix) will be searched by '' | Only the software packages and modules of the currently active tree (denoted by the prefix) will be searched by '' | ||
- | It is easily possible to switch between installation trees with the short commands: | + | It is easily possible to switch between installation trees with the short commands |
< | < | ||
Line 46: | Line 46: | ||
</ | </ | ||
- | This will be required e.g. when software intended to run on GPU nodes needs to be compiled | + | Packages depending on GPU, like '' |
+ | - login on a VSC-5 login node (or a VSC-5 compute node). | ||
+ | - type '' | ||
+ | - compile your code. | ||
< | < | ||
zen [userl51 ~]$ cuz | zen [userl51 ~]$ cuz | ||
- | cuz [userl51 ~]$ | + | cuda-zen |
... build your software ... | ... build your software ... | ||
</ | </ | ||
- | The command | + | The commands '' |
* set PATH | * set PATH | ||
* set MODULEPATH | * set MODULEPATH | ||
Line 60: | Line 63: | ||
* source / | * source / | ||
+ | You can view the shell function '' | ||
---- | ---- | ||
- | ===== Migration script - "spacksearch" ===== | + | ===== Migration script - "spack search" ===== |
- | ==== Method 1 ==== | + | If you need to find a list of packages or modules which correspond to the software you have been using until now, you can use the shell function '' |
- | + | ||
- | If you need to find a list of packages or modules which correspond to the software you have been using until now, you can use the alias **spacksearch** | + | |
< | < | ||
- | alias spacksearch='/ | + | zen [user@l51 ~] spack search |
- | </ | + | </ |
- | The script can be used like in the following example: | + | where '' |
- | + | ||
- | < | + | |
- | zen [user@l51 ~] spacksearch | + | |
- | </code> | + | |
- | where ''< | ||
- | ==== Usage Example | + | ==== Usage example |
- | Let's assume you have used this netcdf-c module: | + | Let's assume you have used this '' |
< | < | ||
Line 95: | Line 92: | ||
</ | </ | ||
- | You can use the migration script to find netcdf-c installations with identical build options: | + | You can use the migration script to find '' |
< | < | ||
- | zen [user@l51 ~]$ spacksearch | + | zen [user@l51 ~]$ spack search |
- | ==> The package | + | ==> The hash ' |
| | ||
- | ==> Searching similar netcdf-c modules | + | ==> Searching similar |
- | + | ||
- | ---/ | + | |
- | netcdf-c/ | + | |
- | netcdf-c/ | + | |
- | netcdf-c/ | + | |
- | netcdf-c/ | + | |
- | netcdf-c/ | + | |
- | netcdf-c/ | + | |
- | netcdf-c/ | + | |
- | netcdf-c/ | + | |
- | ---/gpfs/opt/sw/zen/ | + | -- /opt/sw/skylake/ |
- | netcdf-c/ | + | netcdf-c/ |
+ | netcdf-c/ | ||
+ | netcdf-c/ | ||
+ | netcdf-c/ | ||
- | ==> Load any one of these packages with ' | + | ==> Load any one of these packages with ' |
- | module load netcdf-c/ | + | |
- | ==> Get some additional | + | ==> Get detailed package |
- | spack find -lvd /7qwm3u | + | |
</ | </ | ||
- | To search for packages in a different spack tree you can just change to that tree and then run **spacksearch** | + | To search for packages in a different spack tree you can just change to that tree with '' |
- | For example to search in the ' | + | For example to search in the '' |
< | < | ||
Line 143: | Line 133: | ||
netcdf-c/ | netcdf-c/ | ||
netcdf-c/ | netcdf-c/ | ||
+ | </ | ||
- | ==> Load any one of these packages with ' | + | ---- |
- | module load netcdf-c/4.9.0-gcc-9.5.0-upkxxip | + | |
- | ==> Get some additional info with 'spack find -lvd mypackage', | + | ==== Deprecated: Continue working with old modules (from environments) ==== |
- | spack find -lvd /upkxxi | + | |
- | </ | + | |
+ | <box 80% round red|Warning> | ||
- | ==== Method 2 ==== | + | You may continue to use modules from the spack environments skylake and zen3 by adjusting the '' |
- | Add this function to your '' | + | === On zen3 === |
< | < | ||
- | ## spack command wrapper, so we can include our own `spack something` | + | export MODULEPATH=/ |
- | ## commands, like `spack search`: | + | |
- | spack () { | + | |
- | case "$1" in | + | |
- | " | + | |
- | python3 | + | |
- | ;; | + | |
- | *) | + | |
- | command | + | |
- | esac | + | |
- | } | + | |
</ | </ | ||
- | Usage example: | + | === On skylake === |
< | < | ||
- | skylake [user@l41 ~]$ spack search asd | + | export MODULEPATH=/opt/sw/vsc4/ |
- | + | ||
- | ==> The package hash asd refers to python and belongs to vsc4 as: | + | |
- | | + | |
- | + | ||
- | ==> Searching similar python modules at skylake... | + | |
- | + | ||
- | ---/gpfs/opt/sw/skylake/ | + | |
- | python/3.10.8-gcc-9.5.0-qh22vnd | + | |
- | python/3.10.8-gcc-12.2.0-apbi5uz | + | |
- | python/3.10.8-intel-2021.7.1-p4x6jid | + | |
- | + | ||
- | ==> Load any one of these packages with ' | + | |
- | module load python/3.10.8-intel-2021.7.1-p4x6jid | + | |
- | + | ||
- | ==> Get some additional info with 'spack find -lvd mypackage', | + | |
- | spack find -lvd /p4x6ji | + | |
</ | </ | ||
+ | ---- | ||
- | ===== Continue working with old modules (from environments) ===== | + | ==== Deprecated: |
- | You may continue to use modules from the spack environments skylake and zen3 by adjusting the '' | + | <box 80% round red|Warning>This approach is not recommended |
- | + | ||
- | For '' | + | |
- | <code> | + | |
- | export MODULEPATH=/ | + | |
- | </code> | + | |
- | + | ||
- | For '' | + | |
- | < | + | |
- | export MODULEPATH=/ | + | |
- | </ | + | |
- | + | ||
- | ===== Continue with old spack environments ===== | + | |
You may also continue to work with the spack environments. If you wish to do so you need the following commands: | You may also continue to work with the spack environments. If you wish to do so you need the following commands: | ||
- | == On zen3 == | + | === On zen3 === |
< | < | ||
+ | export SPACK_ROOT=/ | ||
source / | source / | ||
spacktivate zen3 | spacktivate zen3 | ||
</ | </ | ||
- | == On skylake == | + | === On skylake |
< | < | ||
+ | export SPACK_ROOT=/ | ||
source / | source / | ||
spacktivate skylake | spacktivate skylake | ||
</ | </ | ||
- | |||
- | |||
- | <box 80% round red|Warning> | ||
---- | ---- | ||
Line 262: | Line 213: | ||
---- | ---- | ||
- | |||
- |