Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
doku:python [2024/02/22 08:55] – katrin | doku:python [2024/09/18 11:14] (current) – katrin | ||
---|---|---|---|
Line 1: | Line 1: | ||
- | ====== Python | + | ====== Python |
- | Previously we recommended using spack python packages, however we decided to deprecate this approach since it is much easier and user friendly to use ''' | + | Previously we recommended using spack python packages, however we decided to **deprecate** this approach since it is much easier and user friendly to use ''' |
- | The documentation is currently in the process of being revised. For now please either use the old information that is still available OR use the new quickstart guide below. | + | Have a look at the new **quickstart guide** below or the material from the python4hpc training: [[https:// |
- | ===== (Conda) | + | The old approach is now // |
+ | |||
+ | ===== Quickstart: Using Python with Conda ===== | ||
To install a new environment, | To install a new environment, | ||
- | This step is optional but we really recommend it so you and everyone in your team is able to produce the same python environment. | + | This step is optional but we really |
To find valid conda package names look at [[https:// | To find valid conda package names look at [[https:// | ||
Line 18: | Line 20: | ||
channels: | channels: | ||
- conda-forge | - conda-forge | ||
- | - defaults | ||
dependencies: | dependencies: | ||
- python=3.10 | - python=3.10 | ||
Line 36: | Line 37: | ||
# create your environment via environment file | # create your environment via environment file | ||
# this will place the environment in your ~/ | # this will place the environment in your ~/ | ||
- | # note: make sure to use "conda >env< create" | + | # note: |
- | (base) $ conda env create | + | # - make sure to use "conda >env< create" |
+ | # - the name is taken from the yaml file; a custom name can be specified with "-n my-custom-name" | ||
+ | (base) $ conda env create --file my_env.yaml | ||
# after creation you can activate the environment to run code in it | # after creation you can activate the environment to run code in it | ||
- | (base) $ conda activate | + | (base) $ conda activate |
# test python version to make sure we have the right one | # test python version to make sure we have the right one | ||
- | (my-env) $ python --version | + | (my_env) $ python --version |
Python 3.10.11 | Python 3.10.11 | ||
</ | </ | ||
Line 53: | Line 56: | ||
#SBATCH --job-name=slurm_conda_example | #SBATCH --job-name=slurm_conda_example | ||
- | #SBATCH --time 00-00:05:00 | + | #SBATCH --time=00-00:05:00 |
#SBATCH --ntasks=2 | #SBATCH --ntasks=2 | ||
#SBATCH --mem=2GB | #SBATCH --mem=2GB | ||
Line 70: | Line 73: | ||
</ | </ | ||
- | ==== More info about Conda ==== | + | ===== More info about Conda ===== |
For more information about conda check-out the conda notebook of the python4HPC training material: [[https:// | For more information about conda check-out the conda notebook of the python4HPC training material: [[https:// | ||
- | ==== FAQ ==== | + | ===== FAQ ===== |
* **''' | * **''' | ||
Line 80: | Line 83: | ||
Make sure to use the currently provided ''' | Make sure to use the currently provided ''' | ||
- | * **I dont get the right python version and packages** / **I see the error ''' | + | * **I dont get the right python version and packages in my slurm batch file environment** - **What is wrong>? |
- | The default for sbatch is to use the current user environment | + | Make sure that one of the first things in your sbatch |
+ | |||
+ | * **I don't get GPU/CUDA enabled packages when installing a conda environment** / **I get conda installation errors when i select GPU/CUDA enabled builds for my conda environment*** | ||
+ | |||
+ | In order to install packages | ||
+ | < | ||
+ | name: my-pytorch-gpu-env | ||
+ | channels: | ||
+ | - pytorch | ||
+ | - conda-forge | ||
+ | dependencies: | ||
+ | - python=3.12 | ||
+ | - pytorch=2.2.*=*cuda11.8* | ||
+ | </ | ||
+ | |||
+ | < | ||
+ | # for example if you use cuda 11.8 | ||
+ | CONDA_OVERRIDE_CUDA=" | ||
+ | </ | ||
- | Alternatively you can also use sbatch with the ''' | ||
====== Deprecated Information (work in progress) ====== | ====== Deprecated Information (work in progress) ====== | ||
Line 145: | Line 165: | ||
==== Channels ==== | ==== Channels ==== | ||
- | The '' | + | The '' |
Popular channels | Popular channels | ||
Line 155: | Line 175: | ||
If you want to set e.g. '' | If you want to set e.g. '' | ||
<code bash> | <code bash> | ||
+ | conda config –remove channels defaults | ||
conda config --add channels conda-forge | conda config --add channels conda-forge | ||
</ | </ |