Parallel Algorithms on Networks of Processors

Parallel Algorithms on Networks of Processors Roy (Hutch) Pargas, PhD Computer Science (UNC Chapel Hill) School of Computing, Clemson University pargas@clemson.edu

Outline • What are parallel algorithms? Why use them? • Challenges for parallel algorithm designers • Choosing a network • Partitioning the data • Designing the algorithm • Example • Recurrences (binary tree) • Analysis (Speedup and Efficiency) • Summary and Conclusions 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

Why Parallel Computation? 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

New ProcessorsFaster and Cheaper January 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

Partition the Data 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

Organize the Processors 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

Build a Network 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

Choosing a Network 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

Are There Really Any Multiprocessing Systems in Use Today? 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

Are There Really Any Multiprocessing Systems in Use Today? Hamburg June 2011 Top 500 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

SupercomputersNEC/HP Tsubame (Japan) 1.192 petaflops≈ 1.28 quadrillion floating point operations per sec 73,278 Xeon cores Infiniband grid network 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

SupercomputersDawning Nebulae (China) 1.27 petaflops≈ 1.36 quadrillion floating point operations per sec 9280 Intel 6-core Xeon processors = 55,680 cores Infinibandgrid network 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

SupercomputersCray Jaguar (USA) 1.75 petaflops≈ 1.876 quadrillion floating point operations per sec 224,256 AMD cores 3D torus network 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

SupercomputersNUDT Tianhe-1A (China) 2.566 petaflops≈ 2.75 quadrillion floating point operations per sec 14336CPUs Undisclosed proprietary network 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

SupercomputersFujitsu “K” (Japan) K = “kei” = Japanese for 10 quadrillion 8.162 petaflops≈ 9quadrillion floating point operations per sec 68,544 8-core SPARC64 processors = 548,352 cores 3-dimensional torus network called Tofu 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

TOP500 Top 500 Computers in the World 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

Where Does that Leave Us? 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

Where Does that Leave Us? • In a wonderful playground of mathematical algorithmic design where imagination and creativity are key! 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

Challenges for Parallel Algorithm Designers • Choosing a network • Partitioning the problem • Designing the parallel algorithm 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

So Let’s Try It: • Choosing a network • Partitioning the problem • Designing the parallel algorithm 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

So Let’s Try It:Elliptic Partial Diff Eqns • Choosing a network • Partitioning the problem • Designing the parallel algorithm 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

Elliptic PDEs • Problems involving second-order elliptic partial differential equations are equilibrium problems. Given a region R bounded by a curve C and that the unknown function z satisfies Laplace’s or Poisson’s equation in R, the objective is to approximate the value of z at any point in R. The method of finite differences is an often used numerical method for solving this problem. The basic strategy is to approximate the differential equation by a difference equation and to solve the difference equation. 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

Designing the Algorithm • Why Solve Linear Recurrences? 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

Designing the Algorithm • Why Solve Linear Recurrences? • The problem: • Solving PDEs using the Method of Finite Differences leads to 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

Designing the Algorithm • Why Solve Linear Recurrences? • The problem: • Solving PDEs using the Method of Finite Differences leads to • Solving Block Tridiagonal Systems which leads to 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

Designing the Algorithm • Why Solve Linear Recurrences? • The problem: • Solving PDEs using the Method of Finite Differences leads to • Solving Block Tridiagonal Systems which leads to • Solving Tridiagonal Systems which leads to 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

Designing the Algorithm • Why Solve Linear Recurrences? • The problem: • Solving PDEs using the Method of Finite Differences leads to • Solving Block Tridiagonal Systems which leads to • Solving Tridiagonal Systems which leads to • Solving Linear Recurrences 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

Designing the Algorithm • Why Solve Linear Recurrences? • The problem: • Solving PDEs using the Method of Finite Differences leads to • Solving Block Tridiagonal Systems which leads to • Solving Tridiagonal Systems which leads to • Solving Linear Recurrences many many many times 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

Designing the Algorithm • Why Solve Linear Recurrences? • The problem: • Solving PDEs using the Method of Finite Differences leads to • Solving Block Tridiagonal Systems which leads to • Solving Tridiagonal Systems which leads to • Solving Linear Recurrences many many many times • Why Use a Binary Tree? 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

Linear Recurrences • Key Idea: • Successfully solving the original pde problem depends upon solving recurrences quickly and efficiently. 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

Linear Recurrences • Consider the following set of n equations: x0 = a0 x1 = a1 + b1x0 x2= a2+ b2 x1 ... xn-1= an-1+ bn-1 xn-2 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

Linear Recurrences • Consider the following set of n equations: x0 = a0 x1 = a1 + b1x0 x2= a2+ b2 x1 ... xn-1= an-1+ bn-1 xn-2 Can we solve for xi in parallel? 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

Linear Recurrences • For uniformity: x0 = a0+ b0 x-1b0=0, x-1=dummy variable x1 = a1 + b1x0 x2= a2+ b2 x1 ... xn-1= an-1+ bn-1 xn-2 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

Linear Recurrences • For uniformity: x0 = a0+ b0 x-1 x1 = a1 + b1x0 x2= a2+ b2 x1 ... xn-1= an-1+ bn-1 xn-2 • Observe, if xi= a + b xj xj= a’ + b’ xk Then xi = (a + ba’) +bb’ xk = a” + b” xk 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

Linear Recurrences • Notation change x0 = a0+ b0 x-1 C0,-1 = (a0,b0) x1 = a1 + b1x0 C1,0 = (a1,b1) x2= a2+ b2 x1 C2,1 = (a2,b2) ... xn-1= an-1+ bn-1 xn-2 Cn-1,n-2 = (an-1,bn-1) • Observe, if xi= a + b xj xj= a’ + b’ xk Then xi = (a + ba’) +bb’ xk = a” + b” xk 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

Linear Recurrences • Notation change x0 = a0+ b0 x-1 C0,-1 = (a0,b0) x1 = a1 + b1x0 C1,0 = (a1,b1) x2= a2+ b2 x1 C2,1 = (a2,b2) ... xn-1= an-1+ bn-1 xn-2 Cn-1,n-2 = (an-1,bn-1) • Observe, if xi= a + b xjCi,j = (a,b) xj= a’ + b’ xkCj,k = (a’,b’) Then xi = (a + ba’) +bb’ xkCi,jCj,k = = a” + b” xkCi,k = (a+ba’,bb’) 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

Linear Recurrences • To summarize: xi= a + b xjCi,j = (a,b) xj= a’ + b’ xkCj,k = (a’,b’) Then xi = (a + ba’) +bb’ xkCi,jCj,k = = a” + b” xkCi,k = (a+ba’,bb’) 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

Parallel Algorithms on Networks of Processors

Télécharger la présentation

Plus de contenu associé