====== From Serial to Parallel Jobs ======

Parallel computing can significantly enhance the performance of computational tasks by distributing work across multiple processors. Here are two primary methods for parallel programming:

===== Message Passing Interface (MPI) =====

The Message Passing Interface (MPI) is a standardized protocol for communication in distributed computing systems. MPI is designed for scenarios where multiple processes are executed concurrently, working together on a shared problem by exchanging data.

* **MPI Documentation:**
  * [[http://www.mpi-forum.org/docs/mpi-3.0/mpi30-report.pdf|MPI 3.0 Specification]]
  * [[http://www.mpi-forum.org/docs/mpi-2.2/mpi22-report.pdf|MPI 2.2 Specification]]

**Key Points:**
- MPI implementations include MPICH, Open MPI, and others.
- Common programming languages for MPI include **Fortran**, **C**, and **C++**.
- There are also MPI bindings for other languages such as Perl, Python, R, Ruby, Java, and CL.
- The parallel version of **Rstat**, known as **Rmpi**, is available on the cluster.
- **Matlab** provides a parallel computing toolbox with MPI support.
- **Maple** offers grid computing capabilities with MPI protocol support.

===== Open Multi-Processing (OpenMP) =====

OpenMP is designed for multi-platform shared memory multiprocessing, offering compiler directives, library routines, and environment variables to manage parallel execution in C, C++, and Fortran.

**Key Features:**
- OpenMP uses a combination of serial and parallel sections within the code.
- Initially, a serial thread is used, which then spawns several parallel threads.
- Unlike MPI, OpenMP threads share the same memory space, eliminating the need for message passing between threads.

===== Further Information and Training =====

For more information and training on parallel computing, please visit the [[http://typo3.vsc.ac.at/research/vsc-research-center/vsc-school-seminar/|VSC Website]]. The VSC Team regularly organizes courses for both beginners and intermediate users in high-performance computing.

If you have specific questions about your code that cannot be addressed in these courses, you can directly [[doku:contact|contact our team]] for personalized assistance.