Skip to content

User Guide

ScienceApps allows you to interactively run and manage ScienceCluster sessions from the browser.

Interactive Apps

To create a new interactive session, click "Interactive Apps" on the top menu bar then select which App you would like to start.

The following interactive apps are available, where you can analyze data, develop algorithms, and create models:

Beta Interactive Apps

The following apps are available in Beta, so only with very limited support:

Launching a Session

Once you have selected the App, complete the web form to create your session:

  • Project (tenant): (Optional) In most cases, this field should be left blank. If you are a member of multiple research groups, and the cost contribution needs to be assigned to a non-default project, you would then specify the name of the S3IT project that will fund your cost contribution.
  • Hours: How many hours will your interactive session should be available? You can always delete your interactive session at any point to stop the allocation. The maximum duration for a single session is one week (168 hours).
  • Cores: How many CPUs are required?
  • RAM (system memory): How much memory does your session need?
  • Email notifications: Check the box "Receive email on all job state changes" if you want to receive email notifications; for example, when your job starts, fails, or ends.
  • Partition: (Jupyter only). Select the ScienceCluster partition that you want to use. Note: GPUs have their own memory, all of which is allocated to the session, independent of the above RAM setting.

My Interactive Sessions

This gives an overview of currently running interactive Apps. Here you can do the following:

  • Connect to the web-interface of an existing session
  • View and manage queued sessions
  • Delete running sessions to release the allocated resources

Files

You can interact with the filesystem through the web browser.

  • /home is where you store your scripts and small datasets (limited in size).
  • /data is where you store larger datasets and output files.

A full description of the ScienceCluster filesystem is available here. Reminder: Backing up or archiving your files to protect against data loss is the responsibility of the user.

Job Interaction

Here you can view and manage your current cluster jobs (active or in the queue).

With the Job Composer you can create jobs based on templates.

Advanced Topics

Cluster Shells

Start an interactive ssh shell on the frontend node of the cluster, similar to having an interactive session like in this article. You can use this tool to create custom Jupyter kernels as described above.

Note

This will not work on Safari or Internet Explorer. Please use Chrome or Firefox.

Custom Kernels in Jupyter

If you need to install packages or want to use Jupyter with a non-python programming language, you will need to first create a Conda virtual environment. To do this, ssh to the cluster from a terminal, or open an interactive terminal under the Cluster shells section.

Note

We recommend locating your Conda environments in the /data directory. See the warning below for more details.

# Load ScienceCluster modules
module load generic
module load anaconda3

# Create and activate your conda environment
conda create -n myenv
source activate myenv

# Custom package installation goes here, for example:
conda install numpy

# Install the tools to add a custom kernel
conda install ipykernel

# Add your environment to the kernel list
ipython kernel install --user --name myenv

The custom package installation step can also use pip, for example:

conda install pip
pip install pandas

Warning

Only use pip within a Conda environment when the package of interest cannot be installed using conda install. Install all packages that require pip after you've installed all packages that use conda install. Furthermore, you will need a specific version of Python in your Conda environment in order for pip install to work. For additional details on using Conda, see this how-to article.

In case you need to remove a kernel, you can use the following command from within the same Conda environment:

jupyter kernelspec remove myenv

After running these commands, you will be able to switch to your custom kernel when you start an interactive session with a Jupyter Notebook. It's then also possible to install more custom packages directly within the notebook, but please note that it's best practice to use conda install when installing packages into Conda environments.

Warning

Creating custom environments with conda can quickly use up your quota for storage in the /home directory. Conda environments and packages are saved in the hidden .conda folder, accessible at either /home/cluster/$USER/.conda/ or ~/.conda/. From the interactive file explorer, this folder is hidden by default. To access this folder from the interactive file system explorer, click the check box next to "Show Dotfiles". Alternatively, follow the directions in this how-to article on creating a .condarc file to save your Conda environments in the /data directory.


Last update: March 21, 2022