CONTENTS -- Preface -- Chapter 1. Introduction -- 1.1. Definition of Parallel Computing -- 1.2. Evolution of Computers -- 1.3. An Enabling Technology -- 1.4. Cost Effectiveness -- 1.4.1. Purchasing costs -- 1.4.2. Operating costs -- 1.4.3. Programming costs -- Chapter 2. Performance Metrics and Models -- 2.1. Parallel Activity Trace -- 2.2. Speedup -- 2.3. Parallel Efficiency -- 2.4. Load Imbalance -- 2.5. Granularity -- 2.6. Overhead -- 2.7. Scalability -- 2.8. Amdahl's Law -- Chapter 3. Hardware Systems -- 3.1. Node Architectures -- 3.2. Network Interconnections -- 3.2.1. Topology -- 3.2.2. Interconnect Technology -- 3.3. Instruction and Data Streams -- 3.4. Processor-MemoryConnectivity -- 3.5. IO Subsystems -- 3.6. System Convergence -- 3.7. Design Considerations -- Chapter 4. Software Systems -- 4.1. Node Software -- 4.1.1. Operating systems -- 4.1.2. Compilers and libraries -- 4.1.3. Profilers -- 4.2. Programming Models -- 4.2.1. Message passing -- 4.2.2. Shared-memory -- 4.3. ParallelDebuggers -- 4.4. Parallel Profilers -- Chapter 5. Design of Algorithms -- 5.1. Algorithm Models -- 5.1.1. Master-slave -- 5.1.2. Domain decomposition -- 5.1.3. Control decomposition -- 5.1.4. Virtual-shared-memory -- 5.1.5. Comparison of programming models -- 5.1.6. Parallel algorithmic issues -- 5.1.7. Levels of algorithmic complication -- 5.2. Examples of Collective Operations -- 5.2.1. Broadcast -- 5.2.2. Gather and scatter -- 5.2.3. Allgather -- 5.3. Mapping Tasks to Processors -- 5.3.1. Supply matrix -- 5.3.2. Demand matrix -- 5.3.3. Review of mapping models -- 5.3.4. Mapping models and algorithms -- Chapter 6. Linear Algebra -- 6.1. Problem Decomposition -- 6.2. Matrix Operations -- 6.2.1. Matrix-vector multiplications -- 6.2.2. Matrix-matrix multiplications -- 6.3. Solution of Linear Systems -- 6.3.1. Direct methods -- 6.3.2. Iterative methods.

6.3.3. ADI -- Chapter 7. Differential Equations -- 7.1. Integration and Differentiation -- 7.1.1. Riemann summation for integration -- 7.1.2. Monte Carlo method for integration -- 7.1.3. Simple parallelization -- 7.2. Partial Differential Equations -- 7.2.1. Hyperbolic equations -- 7.2.2. 3D Heat equation -- 7.2.3. 2D Poisson equation -- 7.2.4. 3D Poisson equation -- 7.2.5. 3D Helmholtz equation -- 7.2.6. Molecular dynamics -- Chapter 8. Fourier Transforms -- 8.1. Fourier Transforms -- 8.2. Discrete Fourier Transforms -- 8.3. Fast Fourier Transforms -- 8.4. Simple Parallelization -- 8.5. The Transpose Method -- 8.6. Complexity Analysis for FFT -- Chapter 9. Optimization -- 9.1. Monte CarloMethods -- 9.1.1. Basics -- 9.1.2. Metropolis-Hastings Algorithm -- 9.1.3. Simulated annealing -- 9.1.4. Genetic algorithm -- 9.2. Parallelization -- 9.2.1. Basics -- 9.2.2. Chain mixing method -- Chapter 10. Applications -- 10.1. Newton's Equation and Molecular Dynamics -- 10.1.1. Molecular dynamics -- 10.1.2. Basics of classical MD -- 10.2. Schrodinger's Equations and Quantum Mechanics -- 10.3. Partition Function, DFT and Material Science -- 10.3.1. Materials research -- 10.4. Maxwell's Equations and Electrical Engineering -- 10.4.1. Helmholtz equation -- 10.4.2. Electrical engineering -- 10.5. Diffusion Equation and Mechanical Engineering -- 10.6. Navier-Stokes Equation and CFD -- 10.7. Other Applications -- 10.7.1. Astronomy -- 10.7.2. Biological engineering -- 10.7.3. Chemical engineering -- 10.7.4. Geosciences -- 10.7.5. Meteorology -- 10.7.6. Oceanography -- Appendix A. MPI -- A.1. AnMPI Primer -- A.1.1. What is MPI? -- A.1.2. Historical perspective -- A.1.3. Major MPI issues -- A.1.4. Concepts and terminology for message passing -- A.1.5. Using the MPI programming language -- A.1.6. Environment management routines.

A.1.7. Point to point communication routines Blocking message passing routines -- A.1.8. Collective communication routines -- A.2. Examples of Using MPI -- A.2.1. "Hello" from all processes -- A.2.2. Integration -- A.3. MPI Tools -- A.4. Complete List ofMPI Functions -- Appendix B. OpenMP -- B.1. Introduction to OpenMP -- B.1.1. The brief history of OpenMP -- B.2. MemoryModel of OpenMP -- B.3. OpenMP Directives -- B.3.1. Parallel region construct -- B.3.2. Work-sharing constructs -- B.3.3. Directive clauses -- B.4. Synchronization -- B.5. Runtime Library Routines -- B.5.1. Execution environment routines -- B.5.2. Lock routines -- B.5.3. Timing routines -- B.6. Examples of Using OpenMP -- B.6.1. "Hello" from all threads -- B.6.2. Calculating π by integration -- B.7. The Future -- Appendix C. Projects -- Project C.1. Watts and Flops of Supercomputers -- Project C.2. Review of Supercomputers -- Project C.3. Top500 and BlueGene Supercomputers -- Project C.4. Say Hello in Order -- Project C.5. Broadcast on Torus -- Project C.6. Competing with MPI on Broadcast, Scatter, etc -- Project C.7. Simple Matrix Multiplication -- Project C.8. Matrix Multiplication on 4D Torus -- Project C.9. Matrix Multiplication and PAT -- Project C.10. Matrix Inversion -- Project C.11. Simple Analysis of an iBT Network -- Project C.12. Compute Eigenvalues of Adjacency Matrices of Networks -- Project C.13. Mapping Wave Equation to Torus -- Project C.14. Load Balance in 3D Mesh -- Project C.15. Wave Equation and PAT -- Project C.16. Computing Coulomb's Forces -- Project C.17. TimingModel for MD -- Project C.18. Minimizing Lennard-Jones Potential -- Project C.19. Install and Profile CP2K -- Project C.20. Install and Profile CPMD -- Project C.21. Install and Profile NAMD -- Project C.22. FFT on Beowulf -- Project C.23. FFT on BlueGene/Q -- Project C.24. Word Analysis.

Project C.25. Cost Estimate of a 0.1 Pflops System -- Project C.26. Design of a Pflops System -- Appendix D. Program Examples -- D.1. Matrix-Vector Multiplication -- D.2. Long Range N-body Force -- D.3. Integration -- References -- Index.