Créer une présentation
Télécharger la présentation

Télécharger la présentation
## Parallel Algorithms on Networks of Processors

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Parallel Algorithms on Networks of Processors**Roy (Hutch) Pargas, PhD Computer Science (UNC Chapel Hill) School of Computing, Clemson University pargas@clemson.edu**Outline**• What are parallel algorithms? Why use them? • Challenges for parallel algorithm designers • Choosing a network • Partitioning the data • Designing the algorithm • Example • Recurrences (binary tree) • Analysis (Speedup and Efficiency) • Summary and Conclusions 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Why Parallel Computation?**50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Why Parallel Computation?**50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Why Parallel Computation?**50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Why Parallel Computation?**50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Why Parallel Computation?**50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Why Parallel Computation?**50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**New ProcessorsFaster and Cheaper**January 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Partition the Data**50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Organize the Processors**50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Build a Network**50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Build a Network**50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Choosing a Network**50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Choosing a Network**50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Choosing a Network**50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Choosing a Network**50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Choosing a Network**50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Are There Really Any Multiprocessing Systems in Use Today?**50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Are There Really Any Multiprocessing Systems in Use Today?**Hamburg June 2011 Top 500 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**SupercomputersNEC/HP Tsubame (Japan)**1.192 petaflops≈ 1.28 quadrillion floating point operations per sec 73,278 Xeon cores Infiniband grid network 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**SupercomputersDawning Nebulae (China)**1.27 petaflops≈ 1.36 quadrillion floating point operations per sec 9280 Intel 6-core Xeon processors = 55,680 cores Infinibandgrid network 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**SupercomputersCray Jaguar (USA)**1.75 petaflops≈ 1.876 quadrillion floating point operations per sec 224,256 AMD cores 3D torus network 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**SupercomputersNUDT Tianhe-1A (China)**2.566 petaflops≈ 2.75 quadrillion floating point operations per sec 14336CPUs Undisclosed proprietary network 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**SupercomputersFujitsu “K” (Japan)**K = “kei” = Japanese for 10 quadrillion 8.162 petaflops≈ 9quadrillion floating point operations per sec 68,544 8-core SPARC64 processors = 548,352 cores 3-dimensional torus network called Tofu 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**TOP500**Top 500 Computers in the World 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Where Does that Leave Us?**50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Where Does that Leave Us?**• In a wonderful playground of mathematical algorithmic design where imagination and creativity are key! 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Where Does that Leave Us?**• In a wonderful playground of mathematical algorithmic design where imagination and creativity are key! 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Where Does that Leave Us?**• In a wonderful playground of mathematical algorithmic design where imagination and creativity are key! 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Where Does that Leave Us?**• In a wonderful playground of mathematical algorithmic design where imagination and creativity are key! 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Challenges for Parallel Algorithm Designers**• Choosing a network • Partitioning the problem • Designing the parallel algorithm 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**So Let’s Try It:**• Choosing a network • Partitioning the problem • Designing the parallel algorithm 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**So Let’s Try It:Elliptic Partial Diff Eqns**• Choosing a network • Partitioning the problem • Designing the parallel algorithm 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Elliptic PDEs**• Problems involving second-order elliptic partial differential equations are equilibrium problems. Given a region R bounded by a curve C and that the unknown function z satisfies Laplace’s or Poisson’s equation in R, the objective is to approximate the value of z at any point in R. The method of finite differences is an often used numerical method for solving this problem. The basic strategy is to approximate the differential equation by a difference equation and to solve the difference equation. 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Designing the Algorithm**• Why Solve Linear Recurrences? 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Designing the Algorithm**• Why Solve Linear Recurrences? • The problem: • Solving PDEs using the Method of Finite Differences leads to 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Designing the Algorithm**• Why Solve Linear Recurrences? • The problem: • Solving PDEs using the Method of Finite Differences leads to • Solving Block Tridiagonal Systems which leads to 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Designing the Algorithm**• Why Solve Linear Recurrences? • The problem: • Solving PDEs using the Method of Finite Differences leads to • Solving Block Tridiagonal Systems which leads to • Solving Tridiagonal Systems which leads to 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Designing the Algorithm**• Why Solve Linear Recurrences? • The problem: • Solving PDEs using the Method of Finite Differences leads to • Solving Block Tridiagonal Systems which leads to • Solving Tridiagonal Systems which leads to • Solving Linear Recurrences 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Designing the Algorithm**• Why Solve Linear Recurrences? • The problem: • Solving PDEs using the Method of Finite Differences leads to • Solving Block Tridiagonal Systems which leads to • Solving Tridiagonal Systems which leads to • Solving Linear Recurrences many many many times 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Designing the Algorithm**• Why Solve Linear Recurrences? • The problem: • Solving PDEs using the Method of Finite Differences leads to • Solving Block Tridiagonal Systems which leads to • Solving Tridiagonal Systems which leads to • Solving Linear Recurrences many many many times • Why Use a Binary Tree? 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Linear Recurrences**• Key Idea: • Successfully solving the original pde problem depends upon solving recurrences quickly and efficiently. 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Linear Recurrences**• Consider the following set of n equations: x0 = a0 x1 = a1 + b1x0 x2= a2+ b2 x1 ... xn-1= an-1+ bn-1 xn-2 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Linear Recurrences**• Consider the following set of n equations: x0 = a0 x1 = a1 + b1x0 x2= a2+ b2 x1 ... xn-1= an-1+ bn-1 xn-2 Can we solve for xi in parallel? 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Linear Recurrences**• For uniformity: x0 = a0+ b0 x-1b0=0, x-1=dummy variable x1 = a1 + b1x0 x2= a2+ b2 x1 ... xn-1= an-1+ bn-1 xn-2 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Linear Recurrences**• For uniformity: x0 = a0+ b0 x-1 x1 = a1 + b1x0 x2= a2+ b2 x1 ... xn-1= an-1+ bn-1 xn-2 • Observe, if xi= a + b xj xj= a’ + b’ xk Then xi = (a + ba’) +bb’ xk = a” + b” xk 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Linear Recurrences**• Notation change x0 = a0+ b0 x-1 C0,-1 = (a0,b0) x1 = a1 + b1x0 C1,0 = (a1,b1) x2= a2+ b2 x1 C2,1 = (a2,b2) ... xn-1= an-1+ bn-1 xn-2 Cn-1,n-2 = (an-1,bn-1) • Observe, if xi= a + b xj xj= a’ + b’ xk Then xi = (a + ba’) +bb’ xk = a” + b” xk 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Linear Recurrences**• Notation change x0 = a0+ b0 x-1 C0,-1 = (a0,b0) x1 = a1 + b1x0 C1,0 = (a1,b1) x2= a2+ b2 x1 C2,1 = (a2,b2) ... xn-1= an-1+ bn-1 xn-2 Cn-1,n-2 = (an-1,bn-1) • Observe, if xi= a + b xjCi,j = (a,b) xj= a’ + b’ xkCj,k = (a’,b’) Then xi = (a + ba’) +bb’ xkCi,jCj,k = = a” + b” xkCi,k = (a+ba’,bb’) 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines**Linear Recurrences**• To summarize: xi= a + b xjCi,j = (a,b) xj= a’ + b’ xkCj,k = (a’,b’) Then xi = (a + ba’) +bb’ xkCi,jCj,k = = a” + b” xkCi,k = (a+ba’,bb’) 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines