Applied Parallel Programming
CSPP51085
Winter 2011
Professor: Andrew Siegel
Location : Reyerson 277
Time : Tue 5:30-8:20
Office: Ry175-B
Email:asiegel@cs.uchicago.edu
Office Hours: Tue 3:30-5:30 and by appointment
TA:Justin Walker
Office: Ryerson, 4th floor
Email:justinw@uchicago.edu
Study sessions:
Listhost: http://mailman.cs.uchicago.edu/mailman/listinfo/cspp51085
Course Description: Parallel computing allows multiple
processing units to work together simultaneously on a common
task. For certain types of applications, parallelization can increase
execution time in proportion to the number of computers or processors
used. This is a huge advantage for applications which have performance
and/or memory bottlenecks, such as one typically encounters in
financial modelling, physics, engineering, or other applied science
domains. This is a fast-paced applied programming course aimed at
students with significant development experience in either C, C++, or
FORTRAN (Java, Matlab, or Python are also possible, but not ideal). No prior knowledge of
parallel computing is assumed. Students should, however, have both an
interest and some previous experience in either algorithmic
development, numerical methods, applied mathematics, or perhaps any
physics or engineering-type discipline. A brief overview of parallel
computing will be presented at the outset, but the course will be less
on overview of HPC architectures and much more a focus on algorithmic
implementation and performance tuning. The goal of the course it to
give students experience in developing efficient, scalable
(distributed memory) parallel algorithms appropriate for any system
running an implementation of the Message Passing Interface
(MPI). Assignments will be designed with some flexibility to allow
students to explore applying parallel techniques to applications in
their own field of interest.
Course Format: Combination of instructor lecture, student presentations, and group discussion.
Grading: The course grade will be based entirely on the four homework assignments.
Getting Help: When possible, all technical questions should be
discussed over the course listhost (see link above). Otherwise,
please consult TA first and then professor if further assistance is
required.
Suggested texts
- Using MPI
by Gropp, Lusk, and Skjellum
MIT Press
2nd edition
- Numerical Recipes in C (or F77 or F90)
online version here
- MPI: The Complete Reference
Marc Snir, Steve W. Otto, Steven Huss-Lederman, David W. Walker, and Jack Dongarra.
The MIT Press
online version
Required Computing Resources
- An MPI implementation, such as MPICH2, available free here
- A compiler in the language of your choice (everything is available on cs cluster)
- We will get accounts on the ANL BG/P and Full Disclosure parallel machine
Useful web resources
Cheat sheets for class
Lectures
Week 1 Jan 4
- Brief overview of supercomputing: history, vector machines, SMPs, GPGPUs, Top500, etc.
- Installing MPICH
- Running parallel jobs locally on CS cluster
- Overview of initial MPI functions: MPI_init, MPI_Comm_rank, MPI_Comm_size, MPI_Send, MPI_Recv, MPI_Reduce, MPI_Bcast
- Examples: Some basic MPI programs
- Notes
- Assignment 1
Week 2 Jan 11
- Point-to-point communication semantics in depth, blocking vs. non-blocking, send modes, etc.
- Examples of embarrasingly parallel applications: fractals, monte carlo, image processing
- Account setup on ANL cluster
- Examples: Example course programs
- Notes: Summary of lecture
- Assignment: Assignment 2
Week 3 Jan 18
- Non-blocking communication, cont.
- Consideration of solutions to simple dynamic load balancing for Mandelbrot set
- Introduction to heat equation
- Discussion of bc's, ic's
- Examples: Example programs
- Notes: Summary of lecture
Week 4 Jan 25
Week 5 Feb. 1
- Prototype distributed neutral particle simulator
- MADRE Library
-
Week 6 Feb. 8
NO CLASS, Fusion Simulation Program workshop in Sand Diego -- class Rescheduled
Homework: Assignment 4
Week 7 Feb. 15
- MPI Collective functions, cont. (notes from Week 4)
- New application: N-body problem
- Useful techniques for hw3 -Cartesian Communicators, MPI_Send_recv_replace, Parallel grid abstraction
- Examples: Simple in-class programs
- Notes: Summary of lecture
Week 8 Feb. 22
- Tutorial on MPI User Defined Types
- Notes: Summary of lecture
Week 9 March 1
- Parallel linear systems
- Gauss Jordan elimination in serial and parallel
- Tri-diagonal systems in serial and parallel (cyclic reduction)
- Implicit heat equation
- Introduction to PETSc
Week 10 March 8
- GPGPU programming
-
- Final projects