Differences

This shows you the differences between two versions of the page.

--- doku:jupyterhub [2023/04/03 08:23] – [BYOI: Bring your own (Singularity) Image] katrin
+++ doku:jupyterhub [2023/04/03 08:42] – katrin
@@ Line 76: / Line 76: @@
 ==== Necessary Packages ====
-To get your custom image to run in our JupyterHub environment it is necessary to add the following packages into your image
+A good starting point for creating your own container is the documentation of the official JupyterHub Docker Stacks Images @ [[https://github.com/jupyter/docker-stacks|Docker-Stacks]]
+Apart from adding your own software the image also needs to have at least the following packages for it to be able to run in our JupyterHub environment:
 <code>
 # This package pulls in all the necessary dependencies to start a jupyter server
+# Make sure this always matches the current JupyterHub version used by VSC
 jupyterhub==3.1.1
@@ Line 92: / Line 95: @@
 **An up 2 date list of packages can always be found in our repo: [[https://gitlab.tuwien.ac.at/vsc/jupyterhub/jupyterhub-docker-stacks/-/blob/main/tools/requirements.txt|requirements.txt]]**
-==== Via Docker Container ====
+=== Run hooks from /usr/local/bin/before-notebook.d ===
-A good starting point for your own container are the official JupyterHub Docker Stacks Images @ [[https://github.com/jupyter/docker-stacks|Docker-Stacks]]. If you take these as a basis make sure you use the right version for the current version of our JupyterHub
-In order to be able to use the image with JupyterHub you also have to convert it to a **singularity image** (or build it as a singularity image in the first place) and place it into a folder that is accessible from the VSC compute nodes.
-Building a singularity image from e.g. a docker container can easily be done with the `singularity build` command. See the singularity documentation for examples: [[https://apptainer.org/user-docs/3.8/build_a_container.html|Build a (singularity) container]]
-==== Run hooks from /usr/local/bin/before-notebook.d ====
 If your image needs to run hooks before startup (e.g. the pyspark image depends on this [[https://github.com/jupyter/docker-stacks/blob/master/pyspark-notebook/Dockerfile|PySpark Dockerfile]]), the docker stack images provide a folderfor such startup scripts `/usr/local/bin/before-notebook.d`.
@@ Line 110: / Line 105: @@
 Note: Contact us to get read rights to the repository.
+==== Build your own custom image from a docker image ====
+[[https://docs.docker.com/engine/reference/builder/|Dockerfile Documentation]]
+If you already have a docker image you want to start from or if you are more familiar with docker image creation you can just use that image and create it into a singularity image after you are finished.
+Note: Make sure that you use the right versions for the current version of our JupyterHub (the version is displayed at the bottom of the [[http://jupyterhub.vsc.ac.at|jupyterhub page]] and was 3.1.1 at the time of writing).
+After selecting / building the docker image all that needs to be done is to convert it into a **singularity image**.
+You can do this by executing the following lines (assuming your docker image is named "my_image")
+<code>
+singularity build my_image.sif docker://my_image
+</code>
+See the documentation for more examples: [[https://apptainer.org/user-docs/3.8/build_a_container.html|Build a (singularity) container]]
+When the conversion process has finished make sure that the resulting ".sif" file is placed in a folder that is available from all compute nodes (e.g. home dir; data dir)
+Note: the conversion can be done on our cluster since we have singularity installed on our nodes.
+==== Directly build a singularity/apptainer image ====
+Instead of starting with docker you can also directly build a singularity image that is ready to use with our JupyterHub instance.
+For this you need to create a so called ".def" file - see the singularity documentation for more information on this format: [[https://apptainer.org/user-docs/3.8/definition_files.html|Definition Files]]
+Here is a minimal example using the datascience docker stacks image as a basis:
+<code>
+BootStrap: docker
+From: jupyter/datascience-notebook:hub-3.1.1
+%post
+/opt/conda/bin/pip install jupyterhub==3.1.1 git+https://github.com/katringoogoo/batchspawner.git@1.2.0+auth_fix
+</code>
+If we save the file as "my_image.def" we can use the following commands to build the image with singularity
+<code>
+singularity build my_image.sif my_image.def
+</code>