Supported by a fellowship for "Innovations in Digital University Teaching" funded by Stifterverband and the state of NRW, Paderborn University has taken a step towards lowering the the barrier for students and researchers to use high-performance computing resources with Jupyter notebooks. To achieve this goal, the University of Applied Sciences South Westphalia and the Paderborn Center for Parallel Computing at Paderborn University have collaborated on a project with the objective of enabling students to access HPC resources from their web browsers.
With the use of JupyterLab (a next-generation Jupyter notebook Interface) the work is even more flexible. Users can build their environment the way they want, for example by installing JupyterLab extensions or arranging multiple windows side by side, to further enhance their workflow in scientific computing, data science and machine learning. The convenience of accessing computing resources from a web browser in a frictionless way enables users to profit from the computational power of high-performance computers without mastering the complexity of remote system access, Linux, workload management, which is inherent to traditional HPC systems.
The result of the collaboration is an open source software component that supports connecting a JupyterHub to a high-performance computer with the help of various tools. Also HPC users, whose organization does not provide a JupyterHub, have the possibility to build up an environment quickly with the help of results of this project.
Our Noctua high performance computer now has access to a JupyterHub which is integrated with the Slurm workload manager. This gives users the ability to use their web browser to interactively perform computations with access to the file systems on Noctua.
To access the JupyterHub on Noctua uses of PC2 can log in to https://jh.noctua1.pc2.uni-paderborn.de, more documentation can be found in our wiki.
We would like to thank all those, especially the University of Applied Sciences South Westphalia, who have contributed to this project and to the NRW Ministry for Culture and Science and Stifterverband for supporting this project.
Further reading:
Project Jupyter: https://jupyter.org/
digiFellow project: https://www.stifterverband.org/digital-lehrfellows-nrw/2019/giefers_plessl
GitHub Project: https://github.com/pc2/JHub-HPC-Interface/
JupyterHub Noctua: https://jh.noctua1.pc2.uni-paderborn.de
JupyterHub Noctua Documentation: https://uni-paderborn.atlassian.net/wiki/spaces/PC2DOK/pages/12944446/JupyterHub
