OpenACC for Programmers : Concepts and Strategies, First Edition

The Complete Guide to OpenACC for Massively Parallel Programming Scientists and technical professionals can use OpenACC to leverage the immense power of modern GPUs without the complexity traditionally associated with programming them. OpenACCTM for Programmers is one of the first comprehensive and practical overviews of OpenACC for massively parallel programming. This book integrates contributions from 19 leading parallel-programming experts from academia, public research organizations, and industry. The authors and editors explain each key concept behind OpenACC, demonstrate how to use essential OpenACC development tools, and thoroughly explore each OpenACC feature set. Throughout, you'll find realistic examples, hands-on exercises, and case studies showcasing the efficient use of OpenACC language constructs. You'll discover how OpenACC's language constructs can be translated to maximize application performance, and how its standard interface can target multiple platforms via widely used programming languages. Each chapter builds on what you've already learned, helping you build practical mastery one step at a time, whether you're a GPU programmer, scientist, engineer, or student. All example code and exercise solutions are available for download at GitHub. Discover how OpenACC makes scalable parallel programming easier and more practical Walk through the OpenACC spec and learn how OpenACC directive syntax is structured Get productive with OpenACC code editors, compilers, debuggers, and performance analysis tools Build your first real-world OpenACC programs Exploit loop-level parallelism in OpenACC, understand the levels of parallelism available, and maximize accuracy or performance Learn how OpenACC programs are compiled Master OpenACC programming best practices Overcome common performance, portability, and interoperability challenges Efficiently distribute tasks across multiple processors Register your product at informit.com/register for convenient access to downloads, updates, and/or corrections as they become available. Cover......Page 1 Title Page......Page 4 Copyright Page......Page 5 Contents......Page 8 Foreword......Page 16 Preface......Page 22 Acknowledgments......Page 24 About the Contributors......Page 26 Chapter 1: OpenACC in a Nutshell......Page 32 1.1.1 Directives......Page 34 1.1.2 Clauses......Page 35 1.1.3 API Routines and Environment Variables......Page 36 1.2.1 Kernels......Page 37 1.2.3 Loop......Page 39 1.2.4 Routine......Page 40 1.3 The Data Environment......Page 42 1.3.2 Data Clauses......Page 43 1.3.3 The Cache Directive......Page 44 1.3.4 Partial Data Transfers......Page 45 1.5 Exercises......Page 46 Chapter 2: Loop-Level Parallelism......Page 48 2.1 Kernels Versus Parallel Loops......Page 49 2.2 Three Levels of Parallelism......Page 52 2.2.1 Gang, Worker, and Vector Clauses......Page 53 2.2.2 Mapping Parallelism to Hardware......Page 54 2.3.1 Loop Collapse......Page 55 2.3.2 Independent Clause......Page 56 2.3.3 Seq and Auto Clauses......Page 58 2.3.4 Reduction Clause......Page 59 2.4 Summary......Page 61 2.5 Exercises......Page 62 Chapter 3: Programming Tools for OpenACC......Page 64 3.1 Common Characteristics of Architectures......Page 65 3.2 Compiling OpenACC Code......Page 66 3.3 Performance Analysis of OpenACC Applications......Page 67 3.3.1 Performance Analysis Layers and Terminology......Page 68 3.3.2 Performance Data Acquisition......Page 69 3.3.4 The OpenACC Profiling Interface......Page 70 3.3.6 The NVIDIA Profiler......Page 72 3.3.7 The Score-P Tools Infrastructure for Hybrid Applications......Page 75 3.3.8 TAU Performance System......Page 79 3.4 I dentifying Bugs in OpenACC Programs......Page 82 3.5 Summary......Page 84 3.6 Exercises......Page 85 4.1 Case Study......Page 90 4.1.1 Serial Code......Page 92 4.1.2 Compiling the Code......Page 98 4.2.1 Find the Hot Spot......Page 99 4.2.3 OpenACC Implementations......Page 100 4.3 Performance of OpenACC Programs......Page 102 4.4.1 Reducing Data Movement......Page 104 4.4.2 Extra Clever Tweaks......Page 106 4.4.3 Final Result......Page 107 4.5 Summary......Page 109 4.6 Exercises......Page 110 Chapter 5: Compiling OpenACC......Page 112 5.1.1 Parallel Hardware......Page 113 5.1.2 Mapping Loops......Page 114 5.1.3 Memory Hierarchy......Page 116 5.1.4 Reductions......Page 117 5.1.5 OpenACC for Parallelism......Page 118 5.2.1 What Compilers Can Do......Page 119 5.2.2 What Compilers Can’t Do......Page 121 5.3.1 Code Preparation......Page 123 5.3.2 Scheduling......Page 124 5.3.3 Serial Code......Page 125 5.3.4 User Errors......Page 126 5.5 Exercises......Page 128 Chapter 6: Best Programming Practices......Page 132 6.1 General Guidelines......Page 133 6.1.2 Optimizing Data Locality......Page 134 6.2.1 Atomic Operations......Page 136 6.2.2 Kernels and Parallel Constructs......Page 137 6.2.3 Runtime Tuning and the If Clause......Page 138 6.3 Optimize Data Locality......Page 139 6.3.1 Minimum Data Transfer......Page 140 6.3.2 Data Reuse and the Present Clause......Page 141 6.3.4 Array Shaping......Page 142 6.4.1 Background: Thermodynamic Tables......Page 143 6.4.3 Profiling......Page 144 6.4.4 Acceleration with OpenACC......Page 145 6.4.5 Optimized Data Locality......Page 147 6.4.6 Performance Study......Page 148 6.5 Summary......Page 149 6.6 Exercises......Page 150 7.1 Challenges......Page 152 7.2.2 x86_64 Multicore and NVIDIA......Page 154 7.3.1 The OpenACC Memory Model......Page 155 7.3.3 Code Generation......Page 156 7.4 Code Refactoring for Performance Portability......Page 157 7.4.1 HACCmk......Page 158 7.4.2 Targeting Multiple Architectures......Page 159 7.4.4 OpenACC over AMD Bulldozer Multicore......Page 161 7.5 Summary......Page 163 7.6 Exercises......Page 164 8.1 Programming Models......Page 166 8.1.2 OpenMP......Page 169 8.1.4 OpenCL......Page 170 8.1.6 Kokkos......Page 171 8.1.8 Threading Building Blocks......Page 172 8.2 Programming Model Components......Page 173 8.2.1 Parallel Loops......Page 174 8.2.2 Parallel Reductions......Page 176 8.2.3 Tightly Nested Loops......Page 178 8.2.4 Hierarchical Parallelism (Non-Tightly Nested Loops)......Page 180 8.2.5 Task Parallelism......Page 182 8.2.6 Data Allocation......Page 183 8.2.7 Data Transfers......Page 184 8.3 A Case Study......Page 186 8.3.1 Serial Implementation......Page 187 8.3.2 The OpenACC Implementation......Page 188 8.3.3 The OpenMP Implementation......Page 189 8.3.4 The CUDA Implementation......Page 190 8.3.5 The Kokkos Implementation......Page 194 8.3.6 The TBB Implementation......Page 196 8.3.7 Some Performance Numbers......Page 198 8.5 Exercises......Page 201 Chapter 9: OpenACC and Interoperability......Page 204 9.1.1 Example: Image Filtering Using DFTs......Page 205 9.1.2 The host_data Directive and the use_device Clause......Page 208 9.1.3 API Routines for Target Platforms......Page 211 9.2 Calling OpenACC from Native Device Code......Page 212 9.3.1 acc_map_data......Page 213 9.3.2 Calling CUDA Device Routines from OpenACC Kernels......Page 215 9.5 Exercises......Page 216 10.1 Asynchronous Operations......Page 218 10.1.1 Asynchronous OpenACC Programming......Page 221 10.1.2 Software Pipelining......Page 226 10.2.1 Multidevice Pipeline......Page 235 10.2.2 OpenACC and MPI......Page 239 10.4 Exercises......Page 244 11.1 Sunway OpenACC......Page 246 11.1.1 The SW26010 Manycore Processor......Page 247 11.1.2 The Memory Model in the Sunway TaihuLight......Page 248 11.1.3 The Execution Model......Page 249 11.1.4 Data Management......Page 250 11.1.5 Summary......Page 254 11.2.1 The OpenUH Compiler Infrastructure......Page 255 11.2.2 Loop-Scheduling Transformation......Page 257 11.2.3 Performance Evaluation of Loop Scheduling......Page 261 11.2.4 Other Research Topics in OpenUH......Page 265 12.1 A Framework for Directive-Based High-Performance Reconfigurable Computing......Page 268 12.1.1 Introduction......Page 269 12.1.2 Baseline Translation of OpenACC-to-FPGA......Page 270 12.1.3 OpenACC Extensions and Optimization for Efficient FPGA Programming......Page 274 12.1.4 Evaluation......Page 279 12.1.5 Summary......Page 283 12.2 Programming Accelerated Clusters Using XcalableACC......Page 284 12.2.1 Introduction to XcalableMP......Page 285 12.2.2 XcalableACC: XcalableMP Meets OpenACC......Page 288 12.2.3 Omni Compiler Implementation......Page 291 12.2.4 Performance Evaluation on HA-PACS......Page 293 12.2.5 Summary......Page 298 A......Page 300 C......Page 301 D......Page 303 G......Page 305 I......Page 306 L......Page 307 M......Page 308 O......Page 309 P......Page 310 Q......Page 312 S......Page 313 T......Page 314 X......Page 315