Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
Last revisionBoth sides next revision
pandoc:arge_storage:05_backup_on_vsc3:backup_on_vsc3 [2019/05/02 20:03] – Pandoc Auto-commit pandocpandoc:arge_storage:05_backup_on_vsc3:backup_on_vsc3 [2020/10/20 08:09] – Pandoc Auto-commit pandoc
Line 25: Line 25:
 ====== Derzeitiges Off-Site Backup (2) ====== ====== Derzeitiges Off-Site Backup (2) ======
  
-{{:pandoc:arge_storage:05_backup_on_vsc3:backup_on_vsc3:vsc_network.png}}+{{..:vsc_network.png}}
  
 ====== Struktur der Daten ====== ====== Struktur der Daten ======
  
-{{:pandoc:arge_storage:05_backup_on_vsc3:backup_on_vsc3:vsc3_files.png}}+{{..:vsc3_files.png}}
  
 ====== Struktur der Daten (2) ====== ====== Struktur der Daten (2) ======
  
-{{:pandoc:arge_storage:05_backup_on_vsc3:backup_on_vsc3:vsc2_files.png}}+{{..:vsc2_files.png}}
  
 ====== Struktur der Daten (3) ====== ====== Struktur der Daten (3) ======
  
-{{:pandoc:arge_storage:05_backup_on_vsc3:backup_on_vsc3:vsc3_space.png}}+{{..:vsc3_space.png}}
  
 ====== Struktur der Daten (4) ====== ====== Struktur der Daten (4) ======
  
-{{:pandoc:arge_storage:05_backup_on_vsc3:backup_on_vsc3:vsc2_space.png}}+{{..:vsc2_space.png}}
  
 ====== Backup-Maschine ====== ====== Backup-Maschine ======
Line 123: Line 123:
 for i in {1..10} for i in {1..10}
 do  do 
-    rsync -avHAXS --delete src/server${i} dest/server${i} & +    rsync -avHAXS --delete src/server${i}/ dest/server${i}/ & 
 done done
 </code> </code>
Line 150: Line 150:
  
 <code bash> <code bash>
-rsync -avHAXS -f"+ */" -f"- */*" $sourcedir/ . >/dev/null 2>&1 &+# Sync Directory Structure 
 +rsync -avHAXS -f"+ */" -f"- */*" src/server/ dest/server/ & 
 +# Create Filelists
 ( find src/server/ ! -type d > filelist.txt ( find src/server/ ! -type d > filelist.txt
 sort -R filelist.txt > filelist2.txt sort -R filelist.txt > filelist2.txt
 split -d -a6 -n r/$numProcesses filelist2.txt > filelist.scrambled.txt ) & split -d -a6 -n r/$numProcesses filelist2.txt > filelist.scrambled.txt ) &
 +# Delete obsolete files
 rsync -r --delete --existing --ignore-existing src/server/ dest/server/ & rsync -r --delete --existing --ignore-existing src/server/ dest/server/ &
 wait wait
 +# Sync Files in Filelist
 for ((i=0; i<$numProcesses; i++)) for ((i=0; i<$numProcesses; i++))
 do  do 
Line 166: Line 169:
   * Backup mit n-prozessen pro server   * Backup mit n-prozessen pro server
   * Backup Zyklus?   * Backup Zyklus?
-    * Server soll immer eine gewisse load haben+    * Backup-Server soll immer eine gewisse Load haben (Anzahl laufender rsync prozesse)
   * Warteschlange / Queue   * Warteschlange / Queue
-    * Server backuppen, wenn fertig signalisieren sie bereitschaft fuer naechstes backup +    * Backup laeuft pro Server. 
-    * Scheduled script checkt Anzahl der Rsync prozesse 1x pro Stunde+    * Script checkt Server, die nicht gerade gebackuppt werden und fuegt sie in Warteschlange ein 
 +      * 1x Taeglich 
 +    * Zweites script checkt Anzahl der laufenden rsync Prozesse
       * Startet backup fuer naechsten Server in Queue       * Startet backup fuer naechsten Server in Queue
 +      * 1x Stuendlich
  
 ====== Bash/Rsync Magic (6) ====== ====== Bash/Rsync Magic (6) ======
  
-{{:pandoc:arge_storage:05_backup_on_vsc3:backup_on_vsc3:vsc_backup.png}}+{{..:vsc_backup.png}}
  
 ====== Bash/Rsync Magic (7) ====== ====== Bash/Rsync Magic (7) ======
Line 193: Line 199:
   * Hinzufuegen neuer Disks   * Hinzufuegen neuer Disks
   * ZFS Troubles   * ZFS Troubles
 +  * Bottlenecks
 +    * SAS
 +    * Netzwerk
 +    * CPU/Memory
  
 ====== Ausblick ====== ====== Ausblick ======
  
   * Backup auf Tapes mit IBM Spectrum Protect   * Backup auf Tapes mit IBM Spectrum Protect
-    * 10 TB / Tape +    * Tape Library TU Wien Geodaesie 
-    * 200 MB/s write speed per drive+    * 9 TB / Tape unkomprimiert (LTO-7 M8) 
 +    * 300 MB/s write speed per drive
     * 3 Drives     * 3 Drives
     * 100 Tapes     * 100 Tapes
-    * 1 TSM Server +    * 1 Server 
-    * 2 TSM Clients+    * 2 Clients
  
 ====== Ausblick (2) ====== ====== Ausblick (2) ======
  
-  * TSM Server+  * Spectrum Protect Server
     * 2x Intel Xeon E5-2697V4, 2.3 GHz, 18 Cores / 36 Threads     * 2x Intel Xeon E5-2697V4, 2.3 GHz, 18 Cores / 36 Threads
     * 512 GB Ram     * 512 GB Ram
  • pandoc/arge_storage/05_backup_on_vsc3/backup_on_vsc3.txt
  • Last modified: 2020/10/20 09:13
  • by pandoc