MATLAB Parallel Computing ToolboxTM User's Guide

Getting Started Parallel Computing Toolbox Product Description Parallel Computing Support in MathWorks Products Create and Use Distributed Arrays Creating Distributed Arrays Creating Codistributed Arrays Determine Product Installation and Versions Interactively Run a Loop in Parallel Using parfor Run Batch Parallel Jobs Run a Batch Job Run a Batch Job with a Parallel Pool Run Script as Batch Job from the Current Folder Browser Distribute Arrays and Run SPMD Distributed Arrays Single Program Multiple Data (spmd) Composites What Is Parallel Computing? Choose a Parallel Computing Solution Run MATLAB Functions with Automatic Parallel Support Find Automatic Parallel Support Run Non-Blocking Code in Parallel Using parfeval Evaluate Functions in the Background Using parfeval Use Parallel Computing Toolbox with Cloud Center clusters in MATLAB Online Parallel for-Loops (parfor) Decide When to Use parfor parfor-Loops in MATLAB Deciding When to Use parfor Example of parfor With Low Parallel Overhead Example of parfor With High Parallel Overhead Convert for-Loops Into parfor-Loops Ensure That parfor-Loop Iterations are Independent Nested parfor and for-Loops and Other parfor Requirements Nested parfor-Loops Convert Nested for-Loops to parfor-Loops Nested for-Loops: Requirements and Limitations parfor-Loop Limitations Scale Up parfor-Loops to Cluster and Cloud Use parfor-Loops for Reduction Assignments Use Objects and Handles in parfor-Loops Using Objects in parfor-Loops Handle Classes Sliced Variables Referencing Function Handles Troubleshoot Variables in parfor-Loops Ensure That parfor-Loop Variables Are Consecutive Increasing Integers Avoid Overflows in parfor-Loops Solve Variable Classification Issues in parfor-Loops Structure Arrays in parfor-Loops Converting the Body of a parfor-Loop into a Function Unambiguous Variable Names Transparent parfor-loops Global and Persistent Variables Loop Variables Sliced Variables Characteristics of a Sliced Variable Sliced Input and Output Variables Nested for-Loops with Sliced Variables Broadcast Variables Reduction Variables Notes About Required and Recommended Guidelines Basic Rules for Reduction Variables Requirements for Reduction Assignments Using a Custom Reduction Function Chaining Reduction Operators Temporary Variables Uninitialized Temporaries Temporary Variables Intended as Reduction Variables ans Variable Ensure Transparency in parfor-Loops or spmd Statements Parallel Simulink Simulations Improve parfor Performance Where to Create Arrays Profiling parfor-loops Slicing Arrays Optimizing on Local vs. Cluster Workers Run Code on Parallel Pools What Is a Parallel Pool? Automatically Start and Stop a Parallel Pool Alternative Ways to Start and Stop Pools Pool Size and Cluster Selection Choose Between Thread-Based and Process-Based Environments Select Parallel Environment Compare Process Workers and Thread Workers Solve Optimization Problem in Parallel on Thread-Based Pool What Are Thread-Based Environments? What are Process-Based Environments? Check Support for Thread-Based Environment Repeat Random Numbers in parfor-Loops Recommended System Limits for Macintosh and Linux Single Program Multiple Data (spmd) Run Single Programs on Multiple Data Sets Introduction When to Use spmd Define an spmd Statement Display Output MATLAB Path Error Handling spmd Limitations Access Worker Variables with Composites Introduction to Composites Create Composites in spmd Statements Variable Persistence and Sequences of spmd Create Composites Outside spmd Statements Distributing Arrays to Parallel Workers Using Distributed Arrays to Partition Data Across Workers Load Distributed Arrays in Parallel Using datastore Alternative Methods for Creating Distributed and Codistributed Arrays Math with Codistributed Arrays Nondistributed Versus Distributed Arrays Introduction Nondistributed Arrays Codistributed Arrays Working with Codistributed Arrays How MATLAB Software Distributes Arrays Creating a Codistributed Array Local Arrays Obtaining information About the Array Changing the Dimension of Distribution Restoring the Full Array Indexing into a Codistributed Array 2-Dimensional Distribution Looping Over a Distributed Range (for-drange) Parallelizing a for-Loop Codistributed Arrays in a for-drange Loop Run MATLAB Functions with Distributed Arrays Check Distributed Array Support in Functions Support for Sparse Distributed Arrays Programming Overview How Parallel Computing Products Run a Job Overview Toolbox and Server Components Life Cycle of a Job Program a Job on a Local Cluster Specify Your Parallel Preferences Discover Clusters and Use Cluster Profiles Create and Manage Cluster Profiles Discover Clusters Create Cloud Cluster Add and Modify Cluster Profiles Import and Export Cluster Profiles Edit Number of Workers and Cluster Settings Use Your Cluster from MATLAB Apply Callbacks to MATLAB Job Scheduler Jobs and Tasks Job Monitor Typical Use Cases Manage Jobs Using the Job Monitor Identify Task Errors Using the Job Monitor Programming Tips Program Development Guidelines Current Working Directory of a MATLAB Worker Writing to Files from Workers Saving or Sending Objects Using clear functions Running Tasks That Call Simulink Software Using the pause Function Transmitting Large Amounts of Data Interrupting a Job Speeding Up a Job Control Random Number Streams on Workers Client and Workers Different Workers Normally Distributed Random Numbers Profiling Parallel Code Profile Parallel Code Analyze Parallel Profile Data Troubleshooting and Debugging Attached Files Size Limitations File Access and Permissions No Results or Failed Job Connection Problems Between the Client and MATLAB Job Scheduler SFTP Error: Received Message Too Long Big Data Workflow Using Tall Arrays and Datastores Running Tall Arrays in Parallel Use mapreducer to Control Where Your Code Runs Use Tall Arrays on a Parallel Pool Use Tall Arrays on a Spark Enabled Hadoop Cluster Creating and Using Tall Tables Run mapreduce on a Parallel Pool Start Parallel Pool Compare Parallel mapreduce Run mapreduce on a Hadoop Cluster Cluster Preparation Output Format and Order Calculate Mean Delay Partition a Datastore in Parallel Program Independent Jobs Program Independent Jobs Program Independent Jobs on a Local Cluster Create and Run Jobs with a Local Cluster Local Cluster Behavior Program Independent Jobs for a Supported Scheduler Create and Run Jobs Manage Objects in the Scheduler Share Code with the Workers Workers Access Files Directly Pass Data to and from Worker Sessions Pass MATLAB Code for Startup and Finish Plugin Scripts for Generic Schedulers Sample Plugin Scripts Writing Custom Plugin Scripts Adding User Customization Managing Jobs with Generic Scheduler Submitting from a Remote Host Submitting without a Shared File System Program Communicating Jobs Program Communicating Jobs Program Communicating Jobs for a Supported Scheduler Schedulers and Conditions Code the Task Function Code in the Client Further Notes on Communicating Jobs Number of Tasks in a Communicating Job Avoid Deadlock and Other Dependency Errors GPU Computing GPU Capabilities and Performance Capabilities Performance Benchmarking Establish Arrays on a GPU Create GPU Arrays from Existing Data Create GPU Arrays Directly Examine gpuArray Characteristics Save and Load gpuArrays Random Number Streams on a GPU Client CPU and GPU Worker CPU and GPU Normally Distributed Random Numbers Run MATLAB Functions on a GPU MATLAB Functions with gpuArray Arguments Check or Select a GPU Use MATLAB Functions with a GPU Sharpen an Image Using the GPU Compute the Mandelbrot Set using GPU-Enabled Functions Work with Sparse Arrays on a GPU Work with Complex Numbers on a GPU Special Conditions for gpuArray Inputs Acknowledgments Identify and Select a GPU Device Run CUDA or PTX Code on GPU Overview Create a CUDAKernel Object Run a CUDAKernel Complete Kernel Workflow Run MEX-Functions Containing CUDA Code Write a MEX-File Containing CUDA Code Run the Resulting MEX-Functions Comparison to a CUDA Kernel Access Complex Data Compile a GPU MEX-File Measure and Improve GPU Performance Getting Started with GPU Benchmarking Improve Performance Using Single Precision Calculations Basic Workflow for Improving Performance Advanced Tools for Improving Performance Best Practices for Improving Performance Measure Performance on the GPU Vectorize for Improved GPU Performance Troubleshooting GPUs GPU Support by Release Supported GPUs CUDA Toolkit Increase the CUDA Cache Size Objects ClusterPool codistributed codistributor1d codistributor2dbc Composite CUDAKernel distributed Future gpuArray gpuDevice GPUDeviceManager mxGPUArray parallel.Cluster parallel.cluster.Hadoop parallel.gpu.RandStream parallel.Job parallel.Pool parallel.pool.DataQueue parallel.pool.PollableDataQueue parallel.Task parallel.Worker ProcessPool RemoteClusterAccess ThreadPool Functions addAttachedFiles afterAll afterEach afterEach arrayfun batch bsxfun cancel cancel changePassword classUnderlying clear codistributed codistributed.build codistributed.cell codistributed.colon codistributed.spalloc codistributed.speye codistributed.sprand codistributed.sprandn codistributor codistributor1d codistributor1d.defaultPartition codistributor2dbc codistributor2dbc.defaultLabGrid Composite createCommunicatingJob createJob createTask delete delete demote diary distributed distributed.cell distributed.spalloc distributed.speye distributed.sprand distributed.sprandn dload dsave exist existsOnGPU eye false fetchNext fetchOutputs fetchOutputs feval findJob findTask for gather gcat gcp getAttachedFilesFolder getCodistributor getCurrentCluster getCurrentJob getCurrentTask getCurrentWorker getDebugLog getJobClusterData getJobFolder getJobFolderOnCluster getLocalPart getLogLocation getTaskSchedulerIDs globalIndices gop gplus gpuDeviceCount gpurng gputimeit help Inf isaUnderlying iscodistributed isComplete isdistributed isequal isequal isreplicated jobStartup labBarrier labBroadcast labindex labProbe labReceive labSend labSendReceive length listAutoAttachedFiles load logout mapreducer methods mexcuda mpiLibConf mpiprofile mpiSettings mxGPUCopyFromMxArray (C) mxGPUCopyGPUArray (C) mxGPUCopyImag (C) mxGPUCopyReal (C) mxGPUCreateComplexGPUArray (C) mxGPUCreateFromMxArray (C) mxGPUCreateGPUArray (C) mxGPUCreateMxArrayOnCPU (C) mxGPUCreateMxArrayOnGPU (C) mxGPUDestroyGPUArray (C) mxGPUGetClassID (C) mxGPUGetComplexity (C) mxGPUGetData (C) mxGPUGetDataReadOnly (C) mxGPUGetDimensions (C) mxGPUGetNumberOfDimensions (C) mxGPUGetNumberOfElements (C) mxGPUIsSame (C) mxGPUIsSparse (C) mxGPUIsValidGPUData (C) mxGPUSetDimensions (C) mxInitGPU (C) mxIsGPUArray (C) NaN numlabs ones pagefun parallel.cluster.generic.awsbatch.deleteBatchJob parallel.cluster.generic.awsbatch.deleteJobFilesFromS3 parallel.cluster.generic.awsbatch.downloadJobFilesFromS3 parallel.cluster.generic.awsbatch.downloadJobLogFiles parallel.cluster.generic.awsbatch.getBatchJobInfo parallel.cluster.generic.awsbatch.submitBatchJob parallel.cluster.generic.awsbatch.uploadJobFilesToS3 parallel.cluster.Hadoop parallel.clusterProfiles parallel.defaultClusterProfile parallel.exportProfile parallel.gpu.CUDAKernel parallel.gpu.RandStream.create parallel.gpu.RandStream.getGlobalStream parallel.gpu.RandStream.list parallel.gpu.RandStream.setGlobalStream parallel.importProfile parallel.pool.Constant parcluster parfeval parfevalOnAll parfor parforOptions parpool pause pctconfig pctRunDeployedCleanup pctRunOnAll pload pmode poll poolStartup promote psave rand randi randn recreate redistribute reset resume saveAsProfile saveProfile setConstantMemory setJobClusterData shutdown sparse spmd start submit subsasgn subsref taskFinish taskStartup send ticBytes tocBytes true updateAttachedFiles wait wait (cluster) wait wait (GPUDevice) write zeros