Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
pandoc:introduction-to-vsc:01_supercomputers_for_beginners:vsc3_supercomputer [2018/01/31 11:13] – Pandoc Auto-commit pandocpandoc:introduction-to-vsc:01_supercomputers_for_beginners:vsc3_supercomputer [2019/10/08 09:26] (current) – Pandoc Auto-commit pandoc
Line 1: Line 1:
 ====== VSC-3 – supercomputer ====== ====== VSC-3 – supercomputer ======
  
-  * Article written by Claudia Blaas-Schenner (VSC Team) <html><br></html>(last update 2018-01-16 by cb).+  * Article written by Claudia Blaas-Schenner (VSC Team) <html><br></html>(last update 2019-10-08 by cb).
  
 **OUTLINE:** **OUTLINE:**
Line 13: Line 13:
     * VSC-3 compute nodes     * VSC-3 compute nodes
     * VSC-3 node-interconnect     * VSC-3 node-interconnect
-    * VSC-3 ping-pong – intra-node vs. inter-node+    * VSC-3 ping-pong – intra-node vs. inter-node 
  
  
Line 26: Line 27:
   * **VSC is primarily devoted to research.**   * **VSC is primarily devoted to research.**
   * **Who can use VSC?** Scientific personnel of the partner universities, see: http://vsc.ac.at/access <html><nobr></html>VSC is open to users<html></nobr></html> from other academic and research institutions.   * **Who can use VSC?** Scientific personnel of the partner universities, see: http://vsc.ac.at/access <html><nobr></html>VSC is open to users<html></nobr></html> from other academic and research institutions.
-  * **Projects** (test, funded, ...): Access to VSC is granted on the basis of **peer-reviewed projects**. +  * **Projects** (test, funded, …): Access to VSC is granted on the basis of **peer-reviewed projects**. 
-  * **Project manager** (= usually your supervisor): Project application, extensions, creates user accounts, ...+  * **Project manager** (= usually your supervisor): Project application, extensions, creates user accounts, …
   * **Publications**: Please [[http://typo3.vsc.ac.at/access/acknowledgments/|acknowledge VSC]] and [[http://typo3.vsc.ac.at/access/publications-database/|add publications]] <html><font color=#cc3300></html>$~~$➠$~~$<html></font></html> visible on [[http://vsc.ac.at/publications|VSC homepage]] !   * **Publications**: Please [[http://typo3.vsc.ac.at/access/acknowledgments/|acknowledge VSC]] and [[http://typo3.vsc.ac.at/access/publications-database/|add publications]] <html><font color=#cc3300></html>$~~$➠$~~$<html></font></html> visible on [[http://vsc.ac.at/publications|VSC homepage]] !
  
-^VSC links:                                                                                                                                                                  ^Information provided:              ^ 
-|<html><font color=#cc3300></html>➠$~~$<html></font></html>**http://vsc.ac.at**                                                                                              |VSC homepage (general info)        | 
-|<html><font color=#cc3300></html>➠$~~$<html></font></html>**https://service.vsc.ac.at**                                                                                     |VSC service website (application)  | 
-|<html><font color=#cc3300></html>➠$~~$<html></font></html>**https://wiki.vsc.ac.at**                                                                                        |VSC user documentation             | 
-|<html><font color=#cc3300></html>➠$~~$<html></font></html>{{pandoc:introduction-to-vsc:01_supercomputers_for_beginners:vsc3_supercomputer:contact_vsc-red_margin.png?150}}  |VSC user support $~$&$~$ contact   | 
  
 +^VSC links:                                                                                                                                                                   ^Information provided:              ^
 +|<html><font color=#cc3300></html>➠$~~$<html></font></html>**http://vsc.ac.at**                                                                                               |VSC homepage (general info)        |
 +|<html><font color=#cc3300></html>➠$~~$<html></font></html>**https://service.vsc.ac.at**                                                                                      |VSC service website (application)  |
 +|<html><font color=#cc3300></html>➠$~~$<html></font></html>**https://wiki.vsc.ac.at**                                                                                         |VSC user documentation             |
 +|<html><font color=#cc3300></html>➠$~~$<html></font></html>{{:pandoc:introduction-to-vsc:01_supercomputers_for_beginners:vsc3_supercomputer:contact_vsc-red_margin.png?150}}  |VSC user support $~$&$~$ contact   |
  
-  * **VSC Training Courses:** <html><br><font color=#cc3300></html>➠$~~$<html></font></html>**http://vsc.ac.at/training** <html><br></html>**VSC course slides:** <html><br><font color=#cc3300></html>➠$~~$➠$~~$➠$~~$<html></font></html>**[[https://wiki.vsc.ac.at/doku.php?id=pandoc:introduction-to-vsc:01_supercomputers_for_beginners:00_linux|VSC-Linux]]** <html><br><font color=#cc3300></html>➠$~~$➠$~~$➠$~~$<html></font></html>**[[https://wiki.vsc.ac.at/doku.php?id=pandoc:introduction-to-vsc:01_supercomputers_for_beginners:00_intro|VSC-Intro]]**+ 
 +  * **VSC Training Courses:** <html><br></html><html><font color=#cc3300></html>➠$~~$<html></font></html>**http://vsc.ac.at/training** <html><br></html>**VSC course slides:** <html><br></html><html><font color=#cc3300></html>➠$~~$➠$~~$➠$~~$<html></font></html>**[[https://wiki.vsc.ac.at/doku.php?id=pandoc:introduction-to-vsc:01_supercomputers_for_beginners:00_linux|VSC-Linux]]** <html><br></html><html><font color=#cc3300></html>➠$~~$➠$~~$➠$~~$<html></font></html>**[[https://wiki.vsc.ac.at/doku.php?id=pandoc:introduction-to-vsc:01_supercomputers_for_beginners:00_intro|VSC-Intro]]**
  
  
Line 50: Line 52:
  
   * **What is a supercomputer ?**   * **What is a supercomputer ?**
-  * A supercomputer is a computer with a high level of computing performance compared to a general-purpose computer. Performance of a supercomputer is measured in floating-point operations per second (FLOPS)... [from Wikipedia] <html><br><br></html>+  * A supercomputer is a computer with a high level of computing performance compared to a general-purpose computer. Performance of a supercomputer is measured in floating-point operations per second (FLOPS)… [from Wikipedia] <html><br></html><html><br></html>
   * **A supercomputer is listed in the [[https://www.top500.org|TOP500]]**   * **A supercomputer is listed in the [[https://www.top500.org|TOP500]]**
  
Line 57: Line 59:
 |VSC-2 (2011)                                         |  135 TFlop/s|   56 (06/2011)|   71 (06/2011)|    8 PFlop/s #1 (06/2011)| |VSC-2 (2011)                                         |  135 TFlop/s|   56 (06/2011)|   71 (06/2011)|    8 PFlop/s #1 (06/2011)|
 |[[https://www.top500.org/system/178471|VSC-3 (2014)]]|  596 TFlop/s|   85 (11/2014)|   86 (11/2014)|   33 PFlop/s #1 (11/2014)| |[[https://www.top500.org/system/178471|VSC-3 (2014)]]|  596 TFlop/s|   85 (11/2014)|   86 (11/2014)|   33 PFlop/s #1 (11/2014)|
-|VSC-3 (.........)                                    |  596 TFlop/s|  460 (11/2017)|  175 (11/2017)|   93 PFlop/s #1 (11/2017)|+|VSC-3 (………)                                          |  596 TFlop/s|  460 (11/2017)|  175 (11/2017)|   93 PFlop/s #1 (11/2017)| 
 +|[[https://www.top500.org/system/179697|VSC-4 (2019)]]|  2.7 PFlop/s|   82 (06/2019)|            ——–|  148 PFlop/s #1 (06/2019)|
  
  
Line 65: Line 68:
 ====== VSC-3 – what does it look like ? ====== ====== VSC-3 – what does it look like ? ======
  
-{{pandoc:introduction-to-vsc:01_supercomputers_for_beginners:vsc3_supercomputer:vsc3.png}}+{{:pandoc:introduction-to-vsc:01_supercomputers_for_beginners:vsc3_supercomputer:vsc3.png}}
  
  
Line 73: Line 76:
 ====== VSC-3 – what does it look like ? – inside ====== ====== VSC-3 – what does it look like ? – inside ======
  
-{{pandoc:introduction-to-vsc:01_supercomputers_for_beginners:vsc3_supercomputer:vsc3-inside.png}}+{{:pandoc:introduction-to-vsc:01_supercomputers_for_beginners:vsc3_supercomputer:vsc3-inside.png}}
  
  
Line 81: Line 84:
 ====== VSC-3 – components of a supercomputer ====== ====== VSC-3 – components of a supercomputer ======
  
-{{pandoc:introduction-to-vsc:01_supercomputers_for_beginners:vsc3_supercomputer:vsc3-schematic.png}}+{{:pandoc:introduction-to-vsc:01_supercomputers_for_beginners:vsc3_supercomputer:vsc3-schematic.png}}
  
  
Line 95: Line 98:
 ====== Parallel hardware architectures  ====== ====== Parallel hardware architectures  ======
  
-  * **how to connect cores (processing units) ?** <html><br></html>{{pandoc:introduction-to-vsc:01_supercomputers_for_beginners:vsc3_supercomputer:hw-cores_margin.png?150}} 
  
-{{pandoc:introduction-to-vsc:01_supercomputers_for_beginners:vsc3_supercomputer:hw-architectures.png}}+  * **how to connect cores (processing units) ?** <html><br></html>{{:pandoc:introduction-to-vsc:01_supercomputers_for_beginners:vsc3_supercomputer:hw-cores_margin.png?150}} 
 + 
 +{{:pandoc:introduction-to-vsc:01_supercomputers_for_beginners:vsc3_supercomputer:hw-architectures.png}}
  
  
Line 109: Line 113:
  
  
-{{pandoc:introduction-to-vsc:01_supercomputers_for_beginners:vsc3_supercomputer:vsc3-node.png}}+{{:pandoc:introduction-to-vsc:01_supercomputers_for_beginners:vsc3_supercomputer:vsc3-node.png}} 
  
  
Line 128: Line 133:
 ====== VSC-3 node-interconnect ====== ====== VSC-3 node-interconnect ======
  
-**IB fabric = dual rail Intel QDR-80 = 3-level fat-tree** (BF: 2:1 / 4:1) -- schematic figure / numbers only <html><br><font color=#ffffff></html>**IB fabric = dual rail Intel**<html></font></html> (blocking -- BF: down- : up-links -- might introduce an additional latency) 
  
-{{pandoc:introduction-to-vsc:01_supercomputers_for_beginners:vsc3_supercomputer:vsc3-fabric-3.png}}+ 
 + 
 + 
 + 
 + 
 +**IB fabric = dual rail Intel QDR-80 = 3-level fat-tree** (BF: 2:1 / 4:1) – schematic figure / numbers only <html><br></html><html><font color=#ffffff></html>**IB fabric = dual rail Intel **<html></font></html> (blocking – BF: down- : up-links – might introduce an additional latency) 
 + 
 +{{:pandoc:introduction-to-vsc:01_supercomputers_for_beginners:vsc3_supercomputer:vsc3-fabric-3.png}}
  
  
Line 136: Line 147:
 ---- ----
  
-====== VSC-3 ping-pong – intra-node vs. inter-node ======+====== VSC-3 ping-pong – intra-node vs. inter-node ======
  
   * **1 node** $~$ = $~$ 2 sockets with 8 cores per socket $~$ + $~$ **2 HCAs**   * **1 node** $~$ = $~$ 2 sockets with 8 cores per socket $~$ + $~$ **2 HCAs**
Line 142: Line 153:
   * **ping-pong benchmark** $~$ = $~$ module load $~$ intel/16.0.3 $~$ intel-mpi/5.1.3 $~$ | $~$ openmpi/1.10.2 $~$ (1 HCA)   * **ping-pong benchmark** $~$ = $~$ module load $~$ intel/16.0.3 $~$ intel-mpi/5.1.3 $~$ | $~$ openmpi/1.10.2 $~$ (1 HCA)
  
-  * **MPI latency & bandwidth (plus typical values for comparison):** 
  
 +<HTML><ul></HTML>
 +<HTML><li></HTML><HTML><p></HTML>**MPI latency & bandwidth (plus typical values for comparison):**<HTML></p></HTML>
 ^VSC-3:                                                             ^                         latency [μs]                          ^     ^                            typical values for:                             ^                                                       latency^                                                      bandwidth^ ^VSC-3:                                                             ^                         latency [μs]                          ^     ^                            typical values for:                             ^                                                       latency^                                                      bandwidth^
 |<html><font color=#0000ff></html>intra-socket<html></font></html>  |  <html><font color=#0000ff></html>0.3 μs<html></font></html>  |     |       <html><font color=#696969></html>L1 cache<html></font></html>        |   <html><font color=#696969></html>1–2 ns<html></font></html>|  <html><font color=#696969></html>100 GB/s<html></font></html>| |<html><font color=#0000ff></html>intra-socket<html></font></html>  |  <html><font color=#0000ff></html>0.3 μs<html></font></html>  |     |       <html><font color=#696969></html>L1 cache<html></font></html>        |   <html><font color=#696969></html>1–2 ns<html></font></html>|  <html><font color=#696969></html>100 GB/s<html></font></html>|
Line 150: Line 162:
 |<html><font color=#ff00ff></html>IB -2- leaf<html></font></html>   |  <html><font color=#ff00ff></html>1.8 μs<html></font></html>  |     |     <html><font color=#696969></html>HPC networks<html></font></html>      | |<html><font color=#ff00ff></html>IB -2- leaf<html></font></html>   |  <html><font color=#ff00ff></html>1.8 μs<html></font></html>  |     |     <html><font color=#696969></html>HPC networks<html></font></html>      |
 |<html><font color=#ffa500></html>IB -3- spine<html></font></html>  |  <html><font color=#ffa500></html>2.3 μs<html></font></html>  |     |  <html><font color=#696969></html>(per node / 2 HCAs)<html></font></html>  |  <html><font color=#696969></html>1–10 μs<html></font></html>|  <html><font color=#696969></html>1–8 GB/s<html></font></html>| |<html><font color=#ffa500></html>IB -3- spine<html></font></html>  |  <html><font color=#ffa500></html>2.3 μs<html></font></html>  |     |  <html><font color=#696969></html>(per node / 2 HCAs)<html></font></html>  |  <html><font color=#696969></html>1–10 μs<html></font></html>|  <html><font color=#696969></html>1–8 GB/s<html></font></html>|
 +<HTML></li></HTML><HTML></ul></HTML>
 +
 +
  
 +{{:pandoc:introduction-to-vsc:01_supercomputers_for_beginners:vsc3_supercomputer:ping-pong-bandwidth.png}}
  
  
-{{pandoc:introduction-to-vsc:01_supercomputers_for_beginners:vsc3_supercomputer:ping-pong-bandwidth.png}} 
  
 +{{:pandoc:introduction-to-vsc:01_supercomputers_for_beginners:vsc3_supercomputer:ping-pong-bandwidth-log.png}}
  
-{{pandoc:introduction-to-vsc:01_supercomputers_for_beginners:vsc3_supercomputer:ping-pong-bandwidth-log.png}} 
  
  
 ---- ----
  
  • pandoc/introduction-to-vsc/01_supercomputers_for_beginners/vsc3_supercomputer.1517397210.txt.gz
  • Last modified: 2018/01/31 11:13
  • by pandoc