Differences

This shows you the differences between two versions of the page.

--- doku:jupyterhub [2022/05/23 12:32] – katrin
+++ doku:jupyterhub [2023/02/27 09:03] – [BYOI: Bring your own (Singularity) Image] katrin
@@ Line 24: / Line 24: @@
 choose a profile:
   * VSC-4 Singularity Image (this is the default)
+  * VSC-4 python venv
+  * VSC-4 conda python env
   * VSC-3 Singularity Image
-  * VSC-3 GPU nodes - a singularity image with CUDA support (currently only GTX1080)
+  * VSC-3 GTX1080 GPU Singularity Image
-  * VSC-4 python venv - a python virtual environment
+  * VSC-3 A40 GPU Singularity Image
   * If you are participating in a training, there is most likely only a single fixed profile or special training profiles available
@@ Line 73: / Line 76: @@
 A good starting point is to have a look at the official JupyterHub Docker Stacks Images @ [[https://github.com/jupyter/docker-stacks|Docker-Stacks]].
-In a nutshell its necessary to install the `batchspawner` package into your docker image (https://pypi.org/project/batchspawner/) - this package provides functionality needed to run in the slurm environment of VSC (e.g. `batchspawner-singleuser` script).
+In a nutshell it is only necessary to install the following packages into your docker image
+  * jupyterhub==3.0.0 - This package pulls in all the necessary dependencies to start a jupyter server
+  * git+https://github.com/katringoogoo/batchspawner.git@jupyterhub-3.0.0b1 - This package provides functionality needed to run in the slurm environment of VSC (e.g. `batchspawner-singleuser` script).
+In addition we also install the following jupyterlab extensions in our images but they are not strictly necessary and just provide extended functionality like memory monitoring for the user:
+  * jupyterlab-system-monitor
+  * jupyterlab-git
+  * jupyterlab-widgets
+**An up 2 date list of packages can always be found in our repo: [[https://gitlab.tuwien.ac.at/vsc/vsc-jupyterhub-notebooks/-/blob/main/tools/requirements.txt|requirements.txt]]**
+In order to be able to use the image with JupyterHub you also have to make sure to convert it to a **singularity image** (or build it as a singularity image in the first place) and place it into a folder that is accessible from the VSC compute nodes.
+Building a singularity image from e.g. a docker container can easily be done with the `singularity build` command. See the singularity documentation for examples: [[https://apptainer.org/user-docs/3.8/build_a_container.html|Build a (singularity) container]]
-Optionally: If your image needs to run hooks before starting the script the original docker stacks provide a folder for such startup scripts `/usr/local/bin/before-notebook.d` (e.g. the pyspark image depends on this [[https://github.com/jupyter/docker-stacks/blob/master/pyspark-notebook/Dockerfile|PySpark Dockerfile]].
+==== Run hooks from /usr/local/bin/before-notebook.d ====
-Unfortunately the batchspawner package has not been updated yet so in order to restore this functionality we are using an (optional) startscript called `vsc-singleuser.sh`. All it does is to run the hooks from `/usr/local/bin/before-notebook.d` before finally executing the `batchspawner-singleuser` script.
+If your image needs to run hooks before startup (e.g. the pyspark image depends on this [[https://github.com/jupyter/docker-stacks/blob/master/pyspark-notebook/Dockerfile|PySpark Dockerfile]]), the docker stack images provide a folderfor such startup scripts `/usr/local/bin/before-notebook.d`.
-The script can be found at [[https://gitlab.tuwien.ac.at/vsc/vsc-jupyterhub-notebooks/-/blob/main/tools/vsc-singleuser.sh|vsc-singleuser.sh]]. Note: Contact us to get read rights to the repository.
+Unfortunately the batchspawner package does not source them so we are using a custom startscript in our images called `vsc-singleuser.sh`. All it does is to run the hooks from `/usr/local/bin/before-notebook.d` before executing the `batchspawner-singleuser` script.
-In addition we also install the following jupyterlab extensions in our images:
+The script can be found at [[https://gitlab.tuwien.ac.at/vsc/vsc-jupyterhub-notebooks/-/blob/main/tools/vsc-singleuser.sh|vsc-singleuser.sh]].
-* jupyterlab-system-monitor
-* jupyterlab-git
-* jupyterlab-widgets
-An up 2 date list of packages can always be found in our repo: [[https://gitlab.tuwien.ac.at/vsc/vsc-jupyterhub-notebooks/-/blob/main/tools/requirements.txt|requirements.txt]]
+Note: Contact us to get read rights to the repository.
-In order to then use the image with JupyterHub you have to convert it to a singularity image and place it into a folder that is accessible from the VSC compute nodes.
-Building a singularity image from e.g. a docker container can easily be done with the `singularity build` command. See the singularity documentation for examples: [[https://sylabs.io/guides/3.5/user-guide/build_a_container.html|Build a (singularity) container]]