User Guide¶
ScienceApps allows you to interactively run and manage ScienceCluster sessions from the browser.
Interactive Apps¶
To create a new interactive session, click "Interactive Apps" on the top menu bar then select which App you would like to start.
The following interactive apps are available, where you can analyze data, develop algorithms, and create models:
- Jupyter (user documentation)
- RStudio (user documentation)
- Matlab (user documentation)
Beta Interactive Apps¶
The following apps are available in Beta, so only with very limited support:
- Apache Spark (user documentation)
- Code Server (user documentation)
Launching a Session¶
Once you have selected the App, complete the web form to create your session:
- Project (tenant): (Optional) In most cases, this field should be left blank. If you are a member of multiple research groups, and the cost contribution needs to be assigned to a non-default project, you would then specify the name of the Science IT project that will fund your cost contribution.
- Hours: How many hours will your interactive session should be available? You can always delete your interactive session at any point to stop the allocation. The maximum duration for a single session is one week (168 hours).
- Cores: How many CPUs are required?
- RAM (system memory): How much memory does your session need?
- Email notifications: Check the box "Receive email on all job state changes" if you want to receive email notifications; for example, when your job starts, fails, or ends.
- Partition: (Jupyter only). Select the ScienceCluster partition that you want to use. Note: GPUs have their own memory, all of which is allocated to the session, independent of the above RAM setting.
My Interactive Sessions¶
This gives an overview of currently running interactive Apps. Here you can do the following:
- Connect to the web-interface of an existing session
- View and manage queued sessions
- Delete running sessions to release the allocated resources
Session Username and Password¶
You might automatically be signed out from your interactive session.
In case you need to enter the username, use your UZH shortname
.
Warning
The password is NOT your UZH password.
To get the password of a given session, you have to follow the following steps:
- go to apps.s3it.uzh.ch, "My Interactive Sessions" and identify the session that you would like to open
- open the link near the Session ID (you will see several files there)
- click
connection.yml
file (the file will be downloaded on your local machine) - open the downloaded yml file with a file viewer/editor
- use the
password
from the file
Files¶
You can interact with the filesystem through the web browser.
/home
is where you store your scripts and small datasets (limited in size)./data
is where you store larger datasets and output files.
A full description of the ScienceCluster filesystem is available here. Reminder: Backing up or archiving your files to protect against data loss is the responsibility of the user.
Job Interaction¶
Here you can view and manage your current cluster jobs (active or in the queue).
With the Job Composer you can create jobs based on templates.
Advanced Topics¶
Cluster Shells¶
Start an interactive ssh
shell on the frontend node of the cluster, similar to having an interactive session like in this article. You can use this tool to create custom Jupyter kernels as described above.
Note
This will not work on Safari or Internet Explorer. Please use Chrome or Firefox.
Custom Kernels in Jupyter¶
If you need to install packages or want to use Jupyter with a non-python programming language, you will need to first create a Conda virtual environment. To do this, ssh
to the cluster from a terminal, or open an interactive terminal under the Cluster shells section.
Note
We recommend locating your Conda environments in the /data directory
. See the warning below for more details.
# Load ScienceCluster modules
module load anaconda3
# Create and activate your conda environment
conda create --name myenv
source activate myenv
# Custom package installation goes here, for example:
conda install --name myenv numpy
# Install the tools to add a custom kernel
conda install --name myenv ipykernel
# Add your environment to the kernel list
ipython kernel install --user --name myenv
The custom package installation step can also use pip
, for example:
conda install pip
pip install pandas
Warning
Only use pip
within a Conda environment when the package of interest cannot be installed using conda install
. Install all packages that require pip
after you've installed all packages that use conda install
. Furthermore, you will need a specific version of Python in your Conda environment in order for pip install
to work. For additional details on using Conda, see this how-to article.
In case you need to remove a kernel, you can use the following command from within the same Conda environment:
jupyter kernelspec remove myenv
After running these commands, you will be able to switch to your custom kernel when you start an interactive session with a Jupyter Notebook. It's then also possible to install more custom packages directly within the notebook, but please note that it's best practice to use conda install
when installing packages into Conda environments.
Warning
Creating custom environments with conda can quickly use up your quota for storage in the /home
directory. Conda environments and packages are saved in the hidden .conda
folder, accessible at either /home/cluster/$USER/.conda/
or ~/.conda/
. From the interactive file explorer, this folder is hidden by default. To access this folder from the interactive file system explorer, click the check box next to "Show Dotfiles". Alternatively, follow the directions in this how-to article on creating a .condarc
file to save your Conda environments in the /data
directory.
FAQs¶
My ScienceApps homepage won't load. What should I do?¶
If your ScienceApps homepage won't load (e.g., if you receive a message saying "Size of a request header field exceeds server limit."), consider using an incognito/private window in your browser. If you continue to experience trouble with the ScienceApps page not loading, clear your browser cookies and/or your browser cache then reload the page.