Review of Patterns for Parallel Programming by Timothy G. Mattson, Beverly A. Sanders and Berna L. Massingill
Thursday, 01 November 2007
This book gives a broad overview of techniques for writing parallel programs. It is not an API reference, though it does have examples that use OpenMP, MPI and Java, and contains a brief overview of each in appendices. Instead, it covers the issues you have to think about whilst writing parallel programs, starting with identifying the exploitable concurrency in the application, and moving through techniques for structuring algorithms and data, and various synchronization techniques.
The authors do a thorough job of explaining the jargon surrounding parallel programming, such as what a NUMA machine is, what SPMD means, and what makes a program embarrassingly parallel. They also go into some of the more quantitative aspects, like calculating the efficiency of the parallel design, and the serial overhead.
Most of the content is structured in the form of Patterns (hence the title), which I found to be an unusual way of presenting the information. However, the writing is clear, and easily understood. The examples are well though out, and clearly demonstrate the points being made.
The three APIs used for the examples cover the major types of parallel programming environments — explicit threading (Java), message passing (MPI), and implicit threading from high-level constructs (OpenMP). Other threading environments generally fall into one of these categories, so it is usually straightforward to see how descriptions can be extended to other environments for parallel programming.
The authors are clearly coming from a high-performance computing background, with massively parallel computers, but HyperThreading and dual-core CPUs are becoming common on desktops, and many of the same issues apply when writing code to exploit the capabilities of these machines.
Highly Recommended. Everyone writing parallel or multi-threaded programs should read this book.
Buy this book
Patterns for Parallel Programming
Timothy G. Mattson, Beverly A. Sanders and Berna L. Massingill
Published by Addison-Wesley