Differences

This shows you the differences between two versions of the page.

--- pandoc:introduction-to-vsc:02_connecting_to_vsc:connecting_to_vsc [2018/01/31 13:16]
pandoc Pandoc Auto-commit
+++ pandoc:introduction-to-vsc:02_connecting_to_vsc:connecting_to_vsc [2023/03/27 08:22] (current)
goldenberg [Connecting to VSC]
@@ Line 1: / Line 1: @@
 ====== Login and data transfer ======
-  * Article written by Irene Reichl (VSC Team) <html><br></html>(last update 2017-10-10 by ir).
+  * Article written by Irene Reichl (VSC Team) <html><br></html>(last update 2020-10-10 by ir).
+====== Project and User Account ======
-===== Windows =====
+{{.:folie_01_1_connect.png?0x600}}
-[[https://wiki.vsc.ac.at/doku.php?id=doku:win2vsc|How to connect from Windows?]]
+===== Prerequisites =====
-===== Mac =====
+In order to be able to login VSC you need
+  * a user account
+    * ➠ project leader (re-)sets
+      * username
+      * [[https://service.vsc.ac.at/clusteruser/login/|password]]
+  * phone number for receiving OTP
+  * to be inside university network either:
+    * physically
+    * or via VPN-connection to your university
+  * to use a terminal program, e.g. xterm (LINUX), terminal (OSX), putty (originally designed for the Windows platform, however, can also be used on other platforms: https:%%//%%www.putty.org), …
-If, e.g., you have an US keyboard layout, the following lines have to be included in your <html><font color=#cc3300></html>''%%.bashrc%%'' file<html></font></html> on the VSC-3 login node in order to be able to submit jobs:
+====== Project and User Account ======
+{{.:folie_01_3_connect.png?0x150}}
+Let us have a look at the first part, in the figure above denoted by a green box: connecting from the local machine to the remote VSC login nodes.
+----
+====== Connecting to VSC ======
+===== Terminals: =====
+linux: xterm, mac: terminal ## alternative terminal connection <html><font color=#cc3300></html> ''%%PuTTY%%'' <html></font></html> {{.:putty.png?0x670}}
+As stated before, putty was originally designed for the Windows platform, however, can also be used on other platforms, you can download it from: https:%%//%%www.putty.org)
+----
+====== Connecting to VSC ======
+{{.:folie_04_ssh_l31.png?0x600}}
+VSC-4 has 10 named l40, …, l49.
+VSC-5 has 10 named l50, …, l56. (57 to 59 are currently unavailable)
+Login via
 <code bash>
-export LC_CTYPE=en_US.UTF-8
+# VSC-4
-export LC_ALL=en_US.UTF-8
+[...]$ ssh <me>@vsc4.vsc.ac.at
+# VSC-5
+[...]$ ssh <me>@vsc5.vsc.ac.at
 </code>
-Alternatively, you can change the same directly on your Mac in
+assigns you to a VSC-4 oder VSC-5 login node per round robin manner.
-<html><font color=#cc3300></html> Terminal ➠ Preferences ➠ Profiles ➠ Advanced ➠ International <html></font></html>
+In PuTTY you first configure the connection like in the image at the top, and when connecting, you only need to type the username and not the server address too.
-Uncheck the option “Set locale environment variables on startup” and ➠ restart the terminal, see also [[https://wiki.vsc.ac.at/doku.php?id=doku:osx-mavericks&s%5B%5D=locale|OSX: locale settings]].
+**Note:** In the following we use “me” as representative username, please replace that by your own username and omit the “smaller/greater than” signs, i.e., write myusername@vsc……!
-===== Linux and Mac =====
-==== Login ====
-Open a terminal window and type (in OSX you open the program called ''%%terminal%%'' which you can find clicking on the loupe in the upper right corner of your screen and typing ''%%terminal%%'' in the window which opens then.):
+----
+====== Connecting to VSC ======
+{{.:folie_05_ssh_l31.png?0x600}}
-<html><font color=#cc3300></html> VSC-2 <html></font></html>
+You may have reason to login to a specific login server:
 <code bash>
-[...]$ ssh <username>@vsc2.vsc.ac.at
+# VSC-4
+[...]$ ssh <me>@l40.vsc.ac.at
-# or login to dedicated server :
+                ..
-[...]$ ssh <username>@l21.vsc.ac.at
+[...]$ ssh <me>@l49.vsc.ac.at
+# VSC-5
+[...]$ ssh <me>@l50.vsc.ac.at
                 ..
-[...]$ ssh <username>@l23.vsc.ac.at
+[...]$ ssh <me>@l56.vsc.ac.at
 </code>
-<html><font color=#cc3300></html> VSC-3 <html></font></html>
+This may be necessary for trouble shooting.
+----
+----
+====== .bashrc ======
+With the .bashrc file in your home directory you are able to set your preferences for your bash shell. On VSC we provide a recommended .bashrc file that you may use if you like.
+For some libraries on VSC it is important that you have set the language settings properly. A locale is a set of environmental variables that defines the character encoding, language, time/date format, numbers, … for your system libraries and locale-aware applications on a Linux system.
+Maybe in your local
 <code bash>
-[...]$ ssh <username>@vsc3.vsc.ac.at
+/etc/ssh/ssh_config
+</code>
+you find a command
-# or login to dedicated server :
+<code bash>
-[...]$ ssh <username>@l31.vsc.ac.at
+SendEnv LANG LC_*
-                ..
-[...]$ ssh <username>@l35.vsc.ac.at
 </code>
+which means that in ssh connections the local environment is taken to the remote machine. E.g., my local settings are, where some variables are not set as you see.
-The address ''%%<username>@vsc2.vsc.ac.at%%'' points to all **three** login servers of **VSC-2**:, l21, l22, and l23; ''%%<username>@vsc3.vsc.ac.at%%'' points to all **five** login servers of **VSC-3**: l31, l32, l33, l34, and l35. Connections to one of these addresses are assigned to one of the respective servers in round robin manner. The direct login to a dedicated login server can be used for troubleshooting. E.g., in order to resolve specific questions, it may be useful that the support team can test your jobs on the same login server as you.
+<code bash>
+LANG=""
+LC_CTYPE="UTF-8"
+LC_ALL=
+</code>
+That is one point. But even if you do not take your local language settings with you,
+<code>
+has no value on VSC,  by default.
-==== Two factor authentication ====
+Way out:
+For certain libraries it is necessary to set these two lines in your .bashrc:
+# .bashrc
+``` {.bash}
+export LC_CTYPE=en_US.UTF-8
+export LC_ALL=en_US.UTF-8
+</code>
-First, you will be asked to type your password. Then, you will be asked for your **o**ne **t**ime **p**assword (**OTP**; sms token which is sent to your mobile phone). This [[faq:account|OTP]] is an additional feature increasing the security of the cluster.
+----
+----
+====== again and again ======
+{{.:folie_07_oida.png?0x200}}
+The OTP-key holds for 12h and for the IP you login. However, each time you want to have a new shell on the cluster you will be asked to type the password. Is there a way out?
+----
+====== ssh-keys ======
+{{.:folie_08_ssh_keys.png?0x600}}
+Using ssh-keys is a possible which enables to open several shells on the cluster by using a passphrase only once.
+Therefore you generate a key on your local machine which you copy to the remote machine. In order to login you use port 27 instead of the default ssh port 22. The key -optimally and we reaallay ask you to generate it that way- is protected by a passphrase. This passphrase should be as secure as possible.
+----
+====== ssh-keys ======
+{{.:sshkeys_v03.png?0x600|how to set up ssh-keys on client and login node}}
+[[https://wiki.vsc.ac.at/doku.php?id=doku:vpn_ssh_access|WIKI-link: ssh-keys]]
+----
+====== ssh-keys — tunneling ======
+{{.:folie_09_ssh_keys.png?0x600}}
+You may want to jump (tunnel) over a host which is between your local machine and VSC. Then you need to copy your key to both machines, the jump host and VSC.
+----
+====== ssh-keys — tunneling ======
+{{.:sshkeys_3servers_v03.png?0x600|how to set up ssh-keys on client and login node}}
 <code bash>
-[...]$ ssh <username>@l31.vsc.ac.at
+[...]$ eval `ssh-agent`     # start ssh agent
-Password:
+[...]$ echo $SSH_AGENT_SOCK # check if it is running
-sms sent; please enter otp:
+[...]$ chmod 700 ~./ssh     # change rights of .ssh-directioy
+[...]$ ssh-keygen -t rsa    # generate key on local machine
 </code>
+will be asked to create a pass phrase. that passphrase must be as secure as your password. IT IS POSSIBLE AT THAT POINT TO GIVE NO PASSWORD, BUT NEVER EVER GENERATE A KEY WHICH IS NOT PROTECTED !!! This may be a hazard for our system.
+<code bash>
+[...]$ ssh-copy-id <me>@vsc4.vsc.ac.at # copy key to remote machine
+</code>
+will be asked for password & OTP on VSC-4
+more about the [[https://www.ssh.com/ssh/agent|SSH agent]]
-===== Configuration of the ssh login =====
-  * Omitting the port in ''%%ssh <username>@vsc2.vsc.ac.at%%'' or ''%%ssh <username>@vsc3.vsc.ac.at%%'' the default ssh port 22 is taken which always requires password and one-time-password (otp).
-  * Login with ssh-key + otp is only possible at port 27: ''%%ssh -p 27 <username>@vsc2.vsc.ac.at%%'' or ''%%ssh -p 27 <username>@vsc3.vsc.ac.at%%''
-  * Currently, the lifetime of the otp is set to 12 hours, ie. every 12 hours a new otp has to be entered. Logins during the first 12 hours after the last successful login with otp do not require an otp.
+----
+====== Backup Policy ======
+{{.:folie_10_backup.png?0x600}}
-===== LOGIN ➠ terminal in Windows =====
+for details see [[https://service.vsc.ac.at/slides/introduction-to-vsc/08_storage_infrastructure/storage_infrastructure.html#(3)|Storage]]
+----
-===== Data transfer =====
+====== Sync and Copy ======
+{{.:folie_11_scp_rsyn.png?0x200}}
-Transfer your program code, data and input files to your home directory: Please use the latest “Intel” MPI-Environment as described in [[https://wiki.vsc.ac.at/doku.php?id=doku:mpi|MPI Environment]].
+If you copy with scp you always copy everything. On one hand, it may take ages until everything is copied, on the other hand, you may waste storage if there had only been minor modifications in your data. For that reason you may use rsync which first compares source and destination directory/file structure, subsequently only copying what did actually change. This saves time and disc space.
-=== copy data to VSC: ===
+----
+====== Sync and Copy ======
+<code bash>
+scp <source> <destination>
+scp <me>@vsc4.vsc.ac.at:~/<myfile.c> .
+scp myfile.c <me>@vsc4.vsc.ac.at:~/
+</code>
 <code bash>
-[...]$ scp <filename> <username>@vsc2.vsc.ac.at:~/   # VSC-2 OR
+rsync [OPTIONs] <source> <destination>
-[...]$ scp <filename> <username>@vsc3.vsc.ac.at:~/   # VSC-3
+maybe useful: -av, --exclude=/daten/temp/...
 </code>
-==== continous backup of user data  ➠ rsync  ====
+in order to setup on the remote machine, follow: [[https://wiki.vsc.ac.at/doku.php?id=doku:backup&#continous_backup_of_user_data_to_remote_machines|WIKI-link:rsync on remote machines]]
-  * reduces amount of data sent over the network
-    * “quick check” algorithm ➠ only changes sent
-  * [[https://wiki.vsc.ac.at/doku.php?id=doku:backup&#continous_backup_of_user_data_to_remote_machines|rsync options:]] ➠ ''%%rsync -av%%''
-    * recursive, copies symlinks, preserves permissions, modification times, group, special files, owner
+----
-^                            NFS and BeeGFS                             ^                          resists failure of                          ^
+====== Copy — alternative via FileZilla ======
-|  <html><font color=#cc3300></html>RAID-5<html></font></html> (VSC-2)  |  ➠ <html><font color=#cc3300></html> one disk <html></font></html>   |
-|  <html><font color=#cc3300></html>RAID-6<html></font></html> (VSC-3)  |  ➠ <html><font color=#cc3300></html> two disks <html></font></html>  |
-disk controller failure <html><br></html>filesystem software faults <html><br></html><html><font color=blue></html> ➠ might lead to data loss <html></font></html>
+{{.:filezilla_10.png?0x600|FileZilla}}
+For using FileZilla, make sure that you choose “interactive login”. This is necessary to be able to login via two factor authentication. If you are already logged into the cluster via, e.g., PuTTY or another terminal program, you can also login via a normal session because the OTP is only asked once in 12h per cluster (VSC-3 or VSC-4) and IP address.
- Backup Policy: <html><font color=#cc3300></html> no backup of user data on VSC ➠ backup is solely the responsibility of each user <html></font></html>
-Alternatively, data can be copied using, e.g., either //FileZilla// or //winscp//, see figures:
+----
+====== Copy — alternative via winscp ======
+{{.:winscp.png?0x600|winscp}}
-{{:pandoc:introduction-to-vsc:02_connecting_to_vsc:connecting_to_vsc:filezilla_10.png?0x600|FileZilla}}
+----
-{{:pandoc:introduction-to-vsc:02_connecting_to_vsc:connecting_to_vsc:winscp.png?0x600|winscp}}