High-Performance Computing (WS 2016/17)

Goals and Contents

The goal of this course is to teach the fundamentals of high-performance computing. The emphasis of the course is on programming. That is, we will discuss programming models, languages and frameworks for efficiently using parallel computer sytems. The lecture will be complemented by a considerable amount of practical programming exercises that allow the students to gain practical experiences with programming, performance optimization and debugging parallel computer systems. To this end, the student will get access to the HPC clusters operated by the Paderborn Center for Parallel Computing (PC²).

Since the lecture will be taught for the first time in winter term 2016/17, the list of topics is not fixed yet. The lecture and exercises will be based on the textbook Peter S. Pacheo, An introduction to Parallel Programming, Morgan Kaufmann publishers, 2011.

The tentative contents of the lecture are as follows:

1. Fudamentals
   • Brief review of computer architecture (caches, virtual memory, SIMD instructions)
   • Performance Models (Amdahl's law, Gustafson's law, Roofline model, work-span model, strong vs. weak scaling)
   • Parallel Programming Models (shared/distributed memory)
   • Parallel Programming Patterns
2. Performance-aware Single Core Computing
   • Data layout optimization
   • Vectorization
   • Performance optimization with profilers and performance counters
3. Distributed-Memory Programming with Message Passing
   • MPI (Message Passing Interface) fundamentals
   • Collective operations
   • MPI Datatypes
   • Performance analysis with MPI tracing
4. Shared-Memory Programming with Multi-Threading
   • Posix threads
   • OpenMP (work sharing and tasking)
   • Cilk
5. Case Studies / programming exercises (tentative)
   • Dense Linear algebra
   • Finite differences (stencil computation)
   • N-Body simulations
   • Tree Search
   • Rigid body physics simulation
6. Further (optional) topics
   • Heterogeneous computing with GPUs and FPGAs using OpenCL
   • Load balancing (domain decomposition, space filling curves)
   • Visualization with VisIt or ParaView framework