Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
pandoc:introduction-to-vsc:02_connecting_to_vsc:connecting_to_vsc [2018/01/31 11:13]
pandoc Pandoc Auto-commit
pandoc:introduction-to-vsc:02_connecting_to_vsc:connecting_to_vsc [2023/03/27 08:22] (current)
goldenberg [Connecting to VSC]
Line 1: Line 1:
 ====== Login and data transfer ====== ====== Login and data transfer ======
  
-  * Article written by Irene Reichl (VSC Team) <html><br></html>(last update 2017-10-10 by ir).+  * Article written by Irene Reichl (VSC Team) <html><br></html>(last update 2020-10-10 by ir).
  
  
-===== Windows ===== 
  
-[[https://wiki.vsc.ac.at/doku.php?id=doku:win2vsc|How to connect from Windows?]]+====== Project and User Account ======
  
-===== Mac =====+{{.:folie_01_1_connect.png?0x600}}
  
-If, e.g., you have an US keyboard layout, the following lines have to be included in your <html><font color=#cc3300></html>''%%.bashrc%%'' file<html></font></html> on the VSC-3 login node in order to be able to submit jobs:+===== Prerequisites ===== 
 + 
 +In order to be able to login VSC you need  
 +  * a user account  
 +    * ➠ project leader (re-)sets  
 +      * username  
 +      * [[https://service.vsc.ac.at/clusteruser/login/|password]]  
 +  * phone number for receiving OTP  
 +  * to be inside university network either: 
 +    * physically  
 +    * or via VPN-connection to your university  
 +  * to use a terminal program, e.g. xterm (LINUX)terminal (OSX)putty (originally designed for the Windows platform, however, can also be used on other platforms: https:%%//%%www.putty.org), … 
 + 
 + 
 + 
 + 
 +====== Project and User Account ====== 
 + 
 +{{.:folie_01_3_connect.png?0x150}} 
 + 
 +Let us have a look at the first part, in the figure above denoted by a green box: connecting from the local machine to the remote VSC login nodes. 
 + 
 + 
 + 
 +---- 
 + 
 + 
 +====== Connecting to VSC ====== 
 + 
 +===== Terminals: ===== 
 + 
 +linux: xterm, mac: terminal ## alternative terminal connection <html><font color=#cc3300></html> ''%%PuTTY%%'' <html></font></html> {{.:putty.png?0x670}} 
 + 
 +As stated before, putty was originally designed for the Windows platform, however, can also be used on other platforms, you can download it from: https:%%//%%www.putty.org) 
 + 
 + 
 + 
 +---- 
 + 
 +====== Connecting to VSC ====== 
 + 
 +{{.:folie_04_ssh_l31.png?0x600}} 
 + 
 +VSC-4 has 10 named l40, …, l49. 
 + 
 +VSC-5 has 10 named l50, …, l56. (57 to 59 are currently unavailable) 
 + 
 +Login via
  
 <code bash> <code bash>
-export LC_CTYPE=en_US.UTF-8 +# VSC-4 
-export LC_ALL=en_US.UTF-8+[...]$ ssh <me>@vsc4.vsc.ac.at 
 +# VSC-
 +[...]$ ssh <me>@vsc5.vsc.ac.at
 </code> </code>
-Alternatively, you can change the same directly on your Mac in+assigns you to a VSC-4 oder VSC-5 login node per round robin manner.
  
-<html><font color=#cc3300></html> Terminal ➠ Preferences ➠ Profiles ➠ Advanced ➠ International <html></font></html>+In PuTTY you first configure the connection like in the image at the top, and when connecting, you only need to type the username and not the server address too.
  
-Uncheck the option Set locale environment variables on startup” and ➠ restart the terminalsee also [[https://wiki.vsc.ac.at/doku.php?id=doku:osx-mavericks&s%5B%5D=locale|OSX: locale settings]].+**Note:** In the following we use me” as representative username, please replace that by your own username and omit the “smaller/greater than” signsi.e., write myusername@vsc……!
  
-===== Linux and Mac ===== 
  
-==== Login ==== 
  
-Open a terminal window and type (in OSX you open the program called ''%%terminal%%'' which you can find clicking on the loupe in the upper right corner of your screen and typing ''%%terminal%%'' in the window which opens then.):+----
  
 +====== Connecting to VSC ======
  
-<html><font color=#cc3300></html> VSC-2 <html></font></html>+{{.:folie_05_ssh_l31.png?0x600}}
  
-<code bash> +You may have reason to login to a specific login server:
-[...]$ ssh <username>@vsc2.vsc.ac.at+
  
-or login to dedicated server : +<code bash> 
-[...]$ ssh <username>@l21.vsc.ac.at+# VSC-4 
 +[...]$ ssh <me>@l40.vsc.ac.at 
 +                .. 
 +[...]$ ssh <me>@l49.vsc.ac.at 
 +VSC-5 
 +[...]$ ssh <me>@l50.vsc.ac.at
                 ..                 ..
-[...]$ ssh <username>@l23.vsc.ac.at  +[...]$ ssh <me>@l56.vsc.ac.at
 </code> </code>
-<html><font color=#cc3300></html> VSC-3 <html></font></html>+This may be necessary for trouble shooting. 
 + 
 + 
 + 
 +---- 
 + 
 + 
 + 
 +---- 
 + 
 + 
 +====== .bashrc ====== 
 + 
 +With the .bashrc file in your home directory you are able to set your preferences for your bash shell. On VSC we provide a recommended .bashrc file that you may use if you like. 
 + 
 +For some libraries on VSC it is important that you have set the language settings properly. A locale is a set of environmental variables that defines the character encoding, language, time/date format, numbers, … for your system libraries and locale-aware applications on a Linux system. 
 + 
 +Maybe in your local
  
 <code bash> <code bash>
-[...]$ ssh <username>@vsc3.vsc.ac.at+/etc/ssh/ssh_config 
 +</code> 
 +you find a command
  
-# or login to dedicated server : +<code bash
-[...]$ ssh <username>@l31.vsc.ac.at   +SendEnv LANG LC_*
-                .. +
-[...]$ ssh <username>@l35.vsc.ac.at+
 </code> </code>
 +which means that in ssh connections the local environment is taken to the remote machine. E.g., my local settings are, where some variables are not set as you see.
  
-The address ''%%<username>@vsc2.vsc.ac.at%%'' points to all **three** login servers of **VSC-2**:, l21, l22, and l23; ''%%<username>@vsc3.vsc.ac.at%%'' points to all **five** login servers of **VSC-3**: l31, l32, l33, l34, and l35. Connections to one of these addresses are assigned to one of the respective servers in round robin mannerThe direct login to a dedicated login server can be used for troubleshooting. E.g., in order to resolve specific questions, it may be useful that the support team can test your jobs on the same login server as you.+<code bash> 
 +LANG="" 
 +LC_CTYPE="UTF-8" 
 +LC_ALL= 
 +</code> 
 +That is one pointBut even if you do not take your local language settings with you,
  
 +<code>
 +has no value on VSC,  by default.
  
-==== Two factor authentication ====+Way out:  
 +For certain libraries it is necessary to set these two lines in your .bashrc: 
 +# .bashrc 
 +``` {.bash} 
 +export LC_CTYPE=en_US.UTF-8 
 +export LC_ALL=en_US.UTF-8 
 +</code>
  
-First, you will be asked to type your password. Then, you will be asked for your **o**ne **t**ime **p**assword (**OTP**; sms token which is sent to your mobile phone). This [[faq:account|OTP]] is an additional feature increasing the security of the cluster.+ 
 +---- 
 + 
 + 
 + 
 +---- 
 + 
 + 
 +====== again and again ====== 
 + 
 +{{.:folie_07_oida.png?0x200}} 
 + 
 +The OTP-key holds for 12h and for the IP you login. Howevereach time you want to have a new shell on the cluster you will be asked to type the password. Is there a way out? 
 + 
 + 
 + 
 +---- 
 + 
 +====== ssh-keys ====== 
 + 
 +{{.:folie_08_ssh_keys.png?0x600}} 
 + 
 +Using ssh-keys is a possible which enables to open several shells on the cluster by using a passphrase only once. 
 + 
 +Therefore you generate a key on your local machine which you copy to the remote machine. In order to login you use port 27 instead of the default ssh port 22. The key -optimally and we reaallay ask you to generate it that way- is protected by a passphrase. This passphrase should be as secure as possible. 
 + 
 + 
 + 
 +---- 
 + 
 +====== ssh-keys ====== 
 + 
 +{{.:sshkeys_v03.png?0x600|how to set up ssh-keys on client and login node}} 
 + 
 +[[https://wiki.vsc.ac.at/doku.php?id=doku:vpn_ssh_access|WIKI-link: ssh-keys]] 
 + 
 + 
 +---- 
 + 
 +====== ssh-keys — tunneling ====== 
 + 
 +{{.:folie_09_ssh_keys.png?0x600}} 
 + 
 +You may want to jump (tunnel) over a host which is between your local machine and VSC. Then you need to copy your key to both machines, the jump host and VSC. 
 + 
 + 
 + 
 +---- 
 + 
 +====== ssh-keys — tunneling ====== 
 + 
 +{{.:sshkeys_3servers_v03.png?0x600|how to set up ssh-keys on client and login node}}
  
 <code bash> <code bash>
-[...]$ ssh <username>@l31.vsc.ac.at   +[...]$ eval `ssh-agent`     # start ssh agent 
-Password: +[...]$ echo $SSH_AGENT_SOCK # check if it is running 
-sms sent; please enter otp:+[...]$ chmod 700 ~./ssh     # change rights of .ssh-directioy 
 +[...]$ ssh-keygen -t rsa    # generate key on local machine
 </code> </code>
 +will be asked to create a pass phrase. that passphrase must be as secure as your password. IT IS POSSIBLE AT THAT POINT TO GIVE NO PASSWORD, BUT NEVER EVER GENERATE A KEY WHICH IS NOT PROTECTED !!! This may be a hazard for our system.
  
-===== Configuration of the ssh login =====+<code bash> 
 +[...]$ ssh-copy-id <me>@vsc4.vsc.ac.at # copy key to remote machine 
 +</code> 
 +will be asked for password & OTP on VSC-4
  
-  * Omitting the port in ''%%ssh <username>@vsc2.vsc.ac.at%%'' or ''%%ssh <username>@vsc3.vsc.ac.at%%'' the default ssh port 22 is taken which always requires password and one-time-password (otp). +more about the [[https://www.ssh.com/ssh/agent|SSH agent]]
-  * Login with ssh-key + otp is only possible at port 27: ''%%ssh -p 27 <username>@vsc2.vsc.ac.at%%'' or ''%%ssh -p 27 <username>@vsc3.vsc.ac.at%%'' +
-  * Currently, the lifetime of the otp is set to 12 hours, ie. every 12 hours a new otp has to be entered. Logins during the first 12 hours after the last successful login with otp do not require an otp.+
  
  
-===== LOGIN ➠ terminal in Windows ===== 
  
 +----
  
-===== Data transfer =====+====== Backup Policy ======
  
 +{{.:folie_10_backup.png?0x600}}
  
-Transfer your program code, data and input files to your home directory: Please use the latest “Intel” MPI-Environment as described in [[https://wiki.vsc.ac.at/doku.php?id=doku:mpi|MPI Environment]].+for details see [[https://service.vsc.ac.at/slides/introduction-to-vsc/08_storage_infrastructure/storage_infrastructure.html#(3)|Storage]]
  
  
-=== copy data to VSC: ===+---- 
 + 
 + 
 +====== Sync and Copy ====== 
 + 
 +{{.:folie_11_scp_rsyn.png?0x200}} 
 + 
 +If you copy with scp you always copy everything. On one hand, it may take ages until everything is copied, on the other hand, you may waste storage if there had only been minor modifications in your data. For that reason you may use rsync which first compares source and destination directory/file structure, subsequently only copying what did actually change. This saves time and disc space. 
 + 
 + 
 + 
 +---- 
 + 
 +====== Sync and Copy ======
  
 <code bash> <code bash>
-[...]$ scp <filename> <username>@vsc2.vsc.ac.at:~/   # VSC-2 OR +scp <source> <destination> 
-[...]$ scp <filename> <username>@vsc3.vsc.ac.at:~/   # VSC-3+scp <me>@vsc4.vsc.ac.at:~/<myfile.c> . 
 +scp myfile.c <me>@vsc4.vsc.ac.at:~/
 </code> </code>
-==== continous backup of user data  ➠ rsync  ====+<code bash> 
 +rsync [OPTIONs] <source> <destination> 
 +maybe useful: -av, --exclude=/daten/temp/... 
 +</code> 
 +in order to setup on the remote machine, follow: [[https://wiki.vsc.ac.at/doku.php?id=doku:backup&#continous_backup_of_user_data_to_remote_machines|WIKI-link:rsync on remote machines]] 
 + 
 + 
 +----
  
-  * reduces amount of data sent over the network +====== Copy — alternative via FileZilla ======
-    * "quick check" algorithm ➠ only changes sent +
-  * [[https://wiki.vsc.ac.at/doku.php?id=doku:backup&#continous_backup_of_user_data_to_remote_machines|rsync options:]] ➠ ''%%rsync -av%%'' +
-    * recursive, copies symlinks, preserves permissions, modification times, group, special files, owner+
  
 +{{.:filezilla_10.png?0x600|FileZilla}}
  
-^                            NFS and BeeGFS                                                      resists failure of                          ^ +For using FileZilla, make sure that you choose “interactive login”. This is necessary to be able to login via two factor authentication. If you are already logged into the cluster via, e.g., PuTTY or another terminal program, you can also login via a normal session because the OTP is only asked once in 12h per cluster (VSC-3 or VSC-4and IP address.
-|  <html><font color=#cc3300></html>RAID-5<html></font></html> (VSC-2)  |  ➠ <html><font color=#cc3300></html> one disk <html></font></html>   | +
-|  <html><font color=#cc3300></html>RAID-6<html></font></html> (VSC-3  ➠ <html><font color=#cc3300></html> two disks <html></font></html>  |+
  
-disk controller failure <html><br></html>filesystem software faults <html><br><font color=blue></html> ➠ might lead to data loss <html></font></html> 
  
  
- Backup Policy: <html><font color=#cc3300></html> no backup of user data on VSC ➠ backup is solely the responsibility of each user <html></font></html>+----
  
 +====== Copy — alternative via winscp ======
  
-Alternatively, data can be copied using, e.g., either //FileZilla// or //winscp//, see figures:+{{.:winscp.png?0x600|winscp}}
  
  
-{{pandoc:introduction-to-vsc:02_connecting_to_vsc:connecting_to_vsc:filezilla_10.png?0x600}}+----
  
-{{pandoc:introduction-to-vsc:02_connecting_to_vsc:connecting_to_vsc:winscp.png?0x600}} 
  
  • pandoc/introduction-to-vsc/02_connecting_to_vsc/connecting_to_vsc.1517397214.txt.gz
  • Last modified: 2018/01/31 11:13
  • by pandoc