Patterns for Parallel Programming (Software Patterns Series)

Using a patterns approach, this book aims to help software engineers master the challenges of writing software to run on parallel computers. Contents......Page 8 Preface......Page 12 1.1 Introduction......Page 16 1.2 Parallel Programming......Page 18 1.3 Design Patterns and Pattern Languages......Page 19 1.4 A Pattern Language for Parallel Programming......Page 20 2.1 Concurrency in Parallel Programs Versus Operating Systems......Page 22 2.2.1 Flynn’s Taxonomy......Page 23 2.2.2 A Further Breakdown of MIMD......Page 24 2.3 Parallel Programming Environments......Page 27 2.4 The Jargon of Parallel Computing......Page 31 2.5 A Quantitative Look at Parallel Computation......Page 33 2.6.1 Latency and Bandwidth......Page 36 2.6.2 Overlapping Communication and Computation and Latency Hiding......Page 37 2.7 Summary......Page 38 3.1 About the Design Space......Page 39 3.1.1 Overview......Page 40 3.1.3 Background for Examples......Page 41 3.2 The Task Decomposition Pattern......Page 44 3.3 The Data Decomposition Pattern......Page 49 3.4 The Group Tasks Pattern......Page 54 3.5 The Order Tasks Pattern......Page 57 3.6 The Data Sharing Pattern......Page 59 3.7 The Design Evaluation Pattern......Page 64 3.8 Summary......Page 70 4.1 Introduction......Page 72 4.2.1 Target Platform......Page 74 4.2.3 The Algorithm Structure Decision Tree......Page 75 4.3.1 Medical Imaging......Page 77 4.3.2 Molecular Dynamics......Page 78 4.4 The Task Parallelism Pattern......Page 79 4.5 The Divide and Conquer Pattern......Page 88 4.6 The Geometric Decomposition Pattern......Page 94 4.7 The Recursive Data Pattern......Page 112 4.8 The Pipeline Pattern......Page 118 4.9 The Event-Based Coordination Pattern......Page 129 5.1 Introduction......Page 136 5.1.1 Program Structuring Patterns......Page 137 5.2 Forces......Page 138 5.3 Choosing the Patterns......Page 140 5.4 The SPMD Pattern......Page 141 5.5 The Master/Worker Pattern......Page 158 5.6 The Loop Parallelism Pattern......Page 167 5.7 The Fork/Join Pattern......Page 182 5.8 The Shared Data Pattern......Page 188 5.9 The Shared Queue Pattern......Page 198 5.10 The Distributed Array Pattern......Page 213 5.11.1 SIMD......Page 226 5.11.2 MPMD......Page 227 5.11.4 Concurrent Programming with Declarative Languages......Page 229 5.11.5 Problem-Solving Environments......Page 230 6 The Implementation Mechanisms Design Space......Page 231 6.2 UE Management......Page 232 6.2.1 Thread Creation/Destruction......Page 233 6.2.2 Process Creation/Destruction......Page 235 6.3.1 Memory Synchronization and Fences......Page 236 6.3.2 Barriers......Page 241 6.3.3 Mutual Exclusion......Page 244 6.4 Communication......Page 252 6.4.1 Message Passing......Page 253 6.4.2 Collective Communication......Page 260 6.4.3 Other Communication Constructs......Page 266 Appendix A: A Brief Introduction to OpenMP......Page 268 A.1 Core Concepts......Page 269 A.2 Structured Blocks and Directive Formats......Page 272 A.3 Worksharing......Page 274 A.4 Data Environment Clauses......Page 277 A.5 The OpenMP Runtime Library......Page 280 A.6 Synchronization......Page 281 A.7 The Schedule Clause......Page 285 A.8 The Rest of the Language......Page 287 B.1 Concepts......Page 288 B.2 Getting Started......Page 290 B.3 Basic Point-to-Point Message Passing......Page 292 B.4 Collective Operations......Page 294 B.5 Advanced Point-to-Point Message Passing......Page 299 B.6 MPI and Fortran......Page 303 B.7 Conclusion......Page 305 Appendix C: A Brief Introduction to Concurrent Programming in Java......Page 306 C.1 Creating Threads......Page 308 C.3 Synchronized Blocks......Page 312 C.4 Wait and Notify......Page 314 C.5 Locks......Page 316 C.6 Other Synchronization Mechanisms and Shared Data Structures......Page 318 C.7 Interrupts......Page 319 A......Page 322 B......Page 323 C......Page 324 D......Page 325 F......Page 326 J......Page 327 M......Page 328 O......Page 329 P......Page 330 R......Page 331 S......Page 332 T......Page 334 W......Page 335 Bibliography......Page 336 About the Authors......Page 348 B......Page 350 C......Page 351 D......Page 352 E......Page 354 F......Page 355 I......Page 356 J......Page 357 L......Page 358 M......Page 359 N......Page 361 O......Page 362 P......Page 363 S......Page 365 T......Page 368 V......Page 369 W......Page 370

The Parallel Programming Guide for Every Software Developer

From grids and clusters to next-generation game consoles, parallel computing is going mainstream. Innovations such as Hyper-Threading Technology, HyperTransport Technology, and multicore microprocessors from IBM, Intel, and Sun are accelerating the movement's growth. Only one thing is missing: programmers with the skills to meet the soaring demand for parallel software.

That's where Patterns for Parallel Programming comes in. It's the first parallel programming guide written specifically to serve working software developers, not just computer scientists. The authors introduce a complete, highly accessible pattern language that will help any experienced developer "think parallel"-and start writing effective parallel code almost immediately. Instead of formal theory, they deliver proven solutions to the challenges faced by parallel programmers, and pragmatic guidance for using today's parallel APIs in the real world. Coverage includes:

• Understanding the parallel computing landscape and the challenges faced by parallel developers

• Finding the concurrency in a software design problem and decomposing it into concurrent tasks

• Managing the use of data across tasks

• Creating an algorithm structure that effectively exploits the concurrency you've identified

• Connecting your algorithmic structures to the APIs needed to implement them

• Specific software constructs for implementing parallel programs

• Working with today's leading parallel programming environments: OpenMP, MPI, and Java

Patterns have helped thousands of programmers master object-oriented development and other complex programming technologies. With this book, you will learn that they're the best way to master parallel programming too.

0321228111B08232004

This is the eBook version of the printed book. If the print book includes a CD-ROM, this content is not included within the eBook version. The Parallel Programming Guide for Every Software Developer From grids and clusters to next-generation game consoles, parallel computing is going mainstream. Innovations such as Hyper-Threading Technology, HyperTransport Technology, and multicore microprocessors from IBM, Intel, and Sun are accelerating the movement's growth. Only one thing is missing: programmers with the skills to meet the soaring demand for parallel software. That's where Patterns for Parallel Programming comes in. It's the first parallel programming guide written specifically to serve working software developers, not just computer scientists. The authors introduce a complete, highly accessible pattern language that will help any experienced developer'think parallel'-and start writing effective parallel code almost immediately. Instead of formal theory, they deliver proven solutions to the challenges faced by parallel programmers, and pragmatic guidance for using today's parallel APIs in the real world. Coverage includes: Understanding the parallel computing landscape and the challenges faced by parallel developers Finding the concurrency in a software design problem and decomposing it into concurrent tasks Managing the use of data across tasks Creating an algorithm structure that effectively exploits the concurrency you've identified Connecting your algorithmic structures to the APIs needed to implement them Specific software constructs for implementing parallel programs Working with today's leading parallel programming environments: OpenMP, MPI, and Java Patterns have helped thousands of programmers master object-oriented development and other complex programming technologies. With this book, you will learn that they're the best way to master parallel programming too. 0321228111B08232004