Python

Python is comparatively fast evolving programming language, so different versions behave very differently. We provide multiple varieties of python installations, please always use spack to find and load them. On the more recent python versions the following packages are included (eg try import os in python):

• os
• sys
• pathlib
• subprocess

However, even the newer python modules do not contain other widespread packages like numpy. We installed many packages, pease always use spack to find and load them. Yet there are so many additional python packages, some with long dependency chains, that we cannot really install all of them for all the different python versions. All installed python packages are named as py-mypackagename, like py-numpy or py-scipy. If your desired package is not installed, there might be a package at the conda forge.

Sometimes pip is not enough for your use case e.g. when packages need to be compiled and not all packages are available in our infrastructure.

In this case you can use conda (see Anaconda) instead of pip' to set up a consistent local python environment. ==== Setup Conda ==== To load conda on our clusters search for the miniconda3 package. At the time of writing the miniconda3 package is available on both VSC-4 & VSC-5 and can be loaded via <code bash> spack load miniconda3@4.12.0 </code> If you plan to use conda more frequently you can simple add the spack load statement to your ~/.bashrc file to load it automatically after logging in. Also make sure that you run the following statements if you are setting up conda for the first time: <code bash> conda init bash –dry-run –verbose | grep “# »> conda initialize” -A 100 | grep “# «< conda initialize” -B 100 | sed 's/+g' > ~/.bashrc source ~/.bashrc </code> This will add some necessary startup code for conda to your ~/.bashrc file. After executing these steps you will see that your prompt changed to (base) [myname@l51 ~]$ which signifies that conda is active and the base environment is active. ==== Channels ==== The default conda channel points to anacondas repository. However this repo does not always contain the latest packages. There is a community driven channel called conda-forge that has many more packages and most of the time the newer versions readily available. To use conda-forge you need to specify –channel conda-forge when executing conda install commands. If you always want to use conda-forge you can executed the following statement to make it the default for your user <code bash> conda config –add channels conda-forge </code> ==== Create your own custom conda environment ==== In order to create your own user environment you need to do the following steps. To also give a short example for a package which we do not provide via spack we will install phono3py (available onconda forge) into our conda environment (myenv) with conda: <code bash> # create conda env 'myenv', set conda-forge channel as default and use the latest python 3.11 conda create –name myenv –channel conda-forge python=3.11 conda activate myenv conda install –channel conda-forge numpy phono3py </code> With the above statements conda will create a new environment, activate it and install the requested packages into it. You should see that your prompt now changed to (myenv) [myname@l51 ~]$' The following commands provide a bit of introspection to make sure that everything is setup as expected: <code bash> (myenv) [myname@l51 ~]$ which python ~/.conda/envs/myenv/bin/python (myenv) [myname@l51 ~]$ python –version Python 3.11.0 (myenv) [myname@l51 ~]$ which phono3py ~/.conda/envs/myenv/bin/phono3py </code> Starting python in this conda environment (myenv) and loading the packages also works: <code bash> (myenv) myname@l51:~$ python Python 3.11.0 | packaged by conda-forge | (main, Jan 14 2023, 12:27:40) [GCC 11.3.0] on linux Type “help”, “copyright”, “credits” or “license” for more information. »> import numpy »> import phono3py »> exit() </code> ===== SLURM ===== When using slurm it should be enough to include the line conda activate myenv in your slurm batch script. See slurm for detailed information about slurm in general. ===== Jupyterhub ===== See jupyterhub for an even cooler way to access VSC, especially when you are using mainly python.