Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revisionBoth sides next revision | ||
doku:python [2023/02/23 11:13] – [SLURM] katrin | doku:python [2023/03/30 13:16] – [Create conda env from commandline] katrin | ||
---|---|---|---|
Line 38: | Line 38: | ||
To load '' | To load '' | ||
<code bash> | <code bash> | ||
- | spack load miniconda3@4.12.0 | + | # VSC-4 |
+ | module | ||
+ | |||
+ | # VSC-5 | ||
+ | module load miniconda3/ | ||
</ | </ | ||
- | If you plan to use conda more frequently you can simple add the spack load statement to your '' | + | If you plan to use conda more frequently you can simple add the load statement to your '' |
+ | |||
+ | === Optional: execute conda hooks on login === | ||
+ | |||
+ | If you want to have every conda function available directly after logging in you can execute the following statements to add the conda startup code to your '' | ||
- | Also make sure that you run the following statements if you are setting up conda for the **first time**: | ||
<code bash> | <code bash> | ||
conda init bash --dry-run --verbose | grep "# >>> | conda init bash --dry-run --verbose | grep "# >>> | ||
source ~/.bashrc | source ~/.bashrc | ||
</ | </ | ||
- | |||
- | This will add some necessary startup code for conda to your '' | ||
After executing these steps you will see that your prompt changed to '' | After executing these steps you will see that your prompt changed to '' | ||
+ | |||
+ | If you already have an environment you can also add '' | ||
==== Channels ==== | ==== Channels ==== | ||
Line 65: | Line 72: | ||
==== Create your own custom conda environment ==== | ==== Create your own custom conda environment ==== | ||
+ | |||
+ | === Create conda env using environment files === | ||
+ | |||
+ | This is the recommended method to create new conda environments since it makes environment creation reproducible. The file can be easily shared or e.g. added to your version control system | ||
+ | |||
+ | First decide which python version and packages you need. In case you don't know the exact versions upfront you can also just create a first draft of the env-file without pinned version to get the latest libraries. With that information in mind we now write our environment file called " | ||
+ | |||
+ | < | ||
+ | name: myenv | ||
+ | channels: | ||
+ | - conda-forge | ||
+ | dependencies: | ||
+ | - python=3.10 | ||
+ | - pytorch=1.13.1 | ||
+ | </ | ||
+ | |||
+ | After this we run the conda solver to create and install the new environment | ||
+ | |||
+ | < | ||
+ | conda env create -f myenv.yml | ||
+ | </ | ||
+ | |||
+ | Conda will now take its time and solve the environment and then download and install the packages. After this has been done the environment can be activated with | ||
+ | |||
+ | < | ||
+ | conda activate myenv | ||
+ | </ | ||
+ | |||
+ | === Create conda env from commandline === | ||
+ | |||
+ | **Note:** this method is only for illustrating how conda works and is not recommended since it does not create a reproducible environment specification | ||
In order to create your own user environment you need to do the following steps. To also give a short example for a package which we do not provide via spack we will install '' | In order to create your own user environment you need to do the following steps. To also give a short example for a package which we do not provide via spack we will install '' | ||
Line 102: | Line 140: | ||
===== SLURM ===== | ===== SLURM ===== | ||
- | When using slurm it should be enough to include | + | See the following |
<code bash> | <code bash> | ||
- | spack load miniconda3@4.12.0 | + | # |
+ | |||
+ | #SBATCH --job-name=slurm_conda_example | ||
+ | #SBATCH --time 00-00: | ||
+ | #SBATCH --ntasks=2 | ||
+ | #SBATCH --mem=2GB | ||
+ | |||
+ | # modify SBATCH options according to needs | ||
+ | |||
+ | # see "Setup Conda" above or consult " | ||
+ | module | ||
+ | eval " | ||
conda activate myenv | conda activate myenv | ||
+ | |||
+ | # print out some info of the python executable in use | ||
+ | # this should point to the python version from " | ||
+ | which python | ||
+ | python --version | ||
</ | </ | ||
+ | |||
===== JupyterHub ===== | ===== JupyterHub ===== |