Agenda
Introduction to Working on the MUL Cluster
Login
User accounts and passwords are NOT identical to your MUL account
User name = first letter of given name + full surname
example: Erich Mustermann … emustermann
Connections limited to MUL network
Use a terminal program: xterm, putty, terminal, …
Login
Linux and OS X
ssh -X <username>@mul-hpc-81a.unileoben.ac.at
ssh -X <username>@mul-hpc-81b.unileoben.ac.at
OS X
If, e.g., you have an US keyboard layout, you may have to include the following lines have in your <html><font color=#006e6f></html>.bashrc file<html></font></html> on the login node in order to be able to submit jobs:
export LC_CTYPE=en_US.UTF-8
export LC_ALL=en_US.UTF-8
Alternatively, you can change the same directly on your Mac in
<html><font color=#006e6f></html> Terminal ➠ Preferences ➠ Profiles ➠ Advanced ➠ International <html></font></html>
Uncheck the option “Set locale environment variables on startup” and restart the terminal.
Filesystems
We have 3 filesystems for users:
/home … essentially only for config files
NOT for data
NOT for calculations
NOT for compiling codes
/calc … for data (also longer time) and for running calculations
/scratch … fast filesystem for running calculations only
The filesystems are mounted:
via Infiniband on login nodes and compute nodes c1-xx, c2-xx
via Ethernet on all other nodes (c3-xx, c4-xx, c5-xx)
We use ZFS filesystems with these RAID Levels and disk types:
Home directories
home directories are located in /home
mounted via NFS from fileserver f1
For performance reasons /home is separate from /calc
-
Home directories
The directory structure is:
/home/<username>
By default permission are set to 0700:
rwx------
Directories and files are only accessible by user,
no access for group
no access for others.
The settings can be changed by the user who owns the directory.
/calc directories
/calc/<username>
rwx------
purpose of /calc
this is the place for
calculations
compiling codes
data
calculations
/calc quota
/calc has a size of approx. 200 TB
the group quota for /calc depends on your cluster share
every group can have an arbitrary number of users therefore there is no uniform user quota
/scratch directories
/scratch/<username>
rwx------
purpose of /scratch
this is the place for IO heavy calculations
this can be used for software that supports a scratch directory
this is NOT the place for storing files for a longer time
this is also not the place for jobs with almost no IO
/scratch quota
/scratch has a size of approx. 16 TB
there is no need for a quota if you use /scratch like intended:
copy files for a job from /calc to /scratch
run job
copy results of job back to /calc
delete job data immediately from /scratch
the best method is to do the copying and removing in your job script
using /scratch in job script
example (non-working skeleton):
#!/bin/bash
#SBATCH ... see slurm slides ...
[...]
CALC=/home/username/someproject
MYTMP=$(mktemp -p /scratch/username -d)
cp -a $CALC $MYTMP
cd $MYTMP
# run the software here
cp -a $MYTMP/. $CALC/.
rm -rf $MYTMP
Tranferring data
copy data to cluster:
[...]$ scp <filename> <username>@<servername>.unileoben.ac.at:~/
The servername can be one of these:
Tranferring data
backup of data ➠ rsync
reduces amount of data sent over the network
-
recursive, copies symlinks, preserves permissions, modification times, group, special files, owner
Alternatively, data can be copied using, e.g., either <html><font color=#006e6f></html> ➠ FileZilla <html></font></html> or <html><font color=#006e6f></html> ➠ winscp <html></font></html>
Backup Policy: <html><font color=#006e6f></html> no backup of user data ➠ backup is solely the responsibility of each user <html></font></html>
Policies
How to get an account
Policies
Fair share usage
jobs are scheduled in a way that ensures a fair usage of the resources (mostly CPU cores)
there is no static allocation of resources
instead resources are scheduled dynamically
see Background information slides for more technical details
Policies
Getting help
Scratch directories
there is a hierarchy of scratch directories
/dev/shm … RAM disk which uses a maximum of 1/2 of the memory
/tmp … local /tmp directory of the node
/scratch/username … global (on fileserver) scratch directory
you can an use a script like the one shown above to copy/remove data
n.b.: everything you copy into /dev/shm reduces the available memory