Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
doku:backup [2016/06/15 06:20]
markus [Continous Backup of user Data to remote Machines]
doku:backup [2021/09/29 09:31] (current)
goldenberg [Backup]
Line 5: Line 5:
 **Backup is solely the responsibility of each user.** **Backup is solely the responsibility of each user.**
  
-RAID-5 or RAID-6 is used on NFS and FhGFS servers to protect against single disk failure. +RAID-5 or RAID-6 is used on NFS and BeeGFS servers and Disk Pools on the GPFS servers to protect against single disk failure. 
-Up to one (RAID-5) or two (RAID-6) disks failing concurrently can be sustained.+Up to one (RAID-5) or two (RAID-6) or more (Disk Pool) disks failing concurrently can be sustained.
  
 Other failure modes like disk controller failure or filesystem software faults are very seldom but might lead to loss of data. Other failure modes like disk controller failure or filesystem software faults are very seldom but might lead to loss of data.
Line 13: Line 13:
 ======Continous Backup of user Data to remote Machines===== ======Continous Backup of user Data to remote Machines=====
  
-A continuous backup from VSC has to circumvent the entering of the OTP numbers. If it its possible to keep a ssh session alive for a longer time, it can be used to create a tunnel for connecting to a rsync daemon.+A continuous backup from VSC has to circumvent the entering of the OTP numbers. If it its possible to keep a ssh session alive for a longer time, it can be used to create a tunnel for connecting to a rsync daemon. **Each time this tunnel has been disconnected (e.g. reboot of your machine or a VSC login node) you have to re-login with a OTP to reestablish the ssh tunnel**
  
 Do the following steps: Do the following steps:
-  - on the remote machine: <code>+  - on the remote machine, download a script an execute it. It will setup the configuration for a rsync daemon: <code>
 wget https://wiki.vsc.ac.at/lib/exe/fetch.php?media=doku:rsync:rsyncd_setup_remote.sh -O rsyncd_setup_remote.sh wget https://wiki.vsc.ac.at/lib/exe/fetch.php?media=doku:rsync:rsyncd_setup_remote.sh -O rsyncd_setup_remote.sh
 bash rsyncd_setup_remote.sh bash rsyncd_setup_remote.sh
 </code> </code>
-  - On the local machine:<code>+  - On the local machine, also download a script, it will compile the commands for creating the ssh tunnel and using the rsync:<code>
 wget https://wiki.vsc.ac.at/lib/exe/fetch.php?media=doku:rsync:rsyncd_setup_local.sh -O rsyncd_setup_local.sh wget https://wiki.vsc.ac.at/lib/exe/fetch.php?media=doku:rsync:rsyncd_setup_local.sh -O rsyncd_setup_local.sh
 bash rsyncd_setup_local.sh bash rsyncd_setup_local.sh
Line 27: Line 27:
 rsync --daemon --config=/home/ ..../ rsyncd.conf rsync --daemon --config=/home/ ..../ rsyncd.conf
 </code> </code>
-  - open the ssh tunnel 'tunnel_vsc_rsync' on the local machine this will tunnel map a local port to the remote port of the listening rsync daemon. This example maps the local port 5100 to port 8077 on the login node l32: <code> ssh -L 5100:l32.cm.cluster:8077 <user>@l32.vsc.ac.at </code> **(the login node conntected to  must be the same as in step 3)** +  - open the ssh tunnel 'tunnel_vsc_rsync' on the local machine this will tunnel map a local port to the remote port of the listening rsync daemon. This example maps the local port 5100 to port 8077 on the login node l32: <code> ssh -L 5100:l32.cm.cluster:8077 <user>@l32.vsc.ac.at </code> **(the login node "l32" conntected to  must be the same as in step 3)** 
-  - do the 'backup_vsc' on the local machine. This command connects via the local port of the ssh-tunnel to the remote rsync daemon.+  - do the 'backup_vsc' on the local machine. This command connects via the local port of the ssh-tunnel from step 4 to the remote rsync daemon: <code>rsync --password-file=/home/.../rsyncd.secrets -av --port 5100 rsync://<user>@localhost/myfiles /tmp/ </code>
  
  
  
  • doku/backup.1465971651.txt.gz
  • Last modified: 2016/06/15 06:20
  • by markus