Solving [Specific Classes of] Linear Equations using Random Walks

1. Solving [Specific Classes of] Linear Equations using Random Walks Haifeng Qian Sachin Sapatnekar

2. x1 x2 : xn b1 b2 : bn aii Row i = Definition of diagonal dominance • Matrix A is diagonally dominant if aiiij|aij| • Consider a system of linear equations • Can solve Ax = b rapidly using a random walk analogy if A is diagonally dominant (but not singular, of course!) (abbreviated as Ax = b)

3. Why do I care? • Diagonally dominant systems arise in several contexts in CAD (and in other fields) – for example, • Power grid analysis • VLSI Placement • ESD analysis • Thermal analysis • FEM/FDM analyses • Potential theory • The idea for random walk-based linear solvers has been around even in the popular literature • R. Hersh and R. J. Griego, “Brownian motion and potential theory,” Scientific American, pp. 67-74, March 1969. • P. G. Doyle and J. L. Snell, Random walks and electrical networks, Mathematical Association of America, Washington DC, 1984. http://math.dartmouth.edu/~doyle/docs/walks/walks.pdf

4. Dirichlet’s problem • Dirichlet problem: an example – thermal analysis • Given a body of arbitrary shape, and complete information about temperature on the surface: find termperature at an internal point • Temperature is a harmonic function: temperature at a point depends on average temperature of surrounding points • Shizuo Kakutani (1944) – solution of Dirichlet problem • Brownian motion starting from a point (say, a) • Take an award T = temperature of first boundary point hit • Find E[T] a b

5. A Direct Solver

6. Equations Random event(s) Variables Expectation of random variables Approximate solution Average of random samples Stochastic solvers Stochastic solver methodology

7. 2 g2 g1 g3 x 3 1 g4 Ix 4 Example: Power grid analysis • Network of resistors and current sources • The equation at node x is Alternatively

8. ∑ 1 Mapping this to the random walk “game” • Solving a grid amounts to solving, at each node:or

9. $ Home Home Home Random walk overview • Given: • A network of roads • A motel at eachintersection • A set of homes • Random walk • Walk one (randomlychosen) road every day • Stay the night at a motel (pay for it!) • Keep going until reaching home • Win a reward for reaching home! • Problem: find the expected amount of money in the end as a function of the starting node x

10. 4 px,4 x 1 3 px,1 px,3 px,2 2 Random walk overview • For every node

11. Linear equation set Random walk game M walks from i-th node Take average This yields xi Random walk overview

12. $0 $1 0.33 0.33 +$0.13 -$0.2 -$ 0.05 +$ 0.45 A A 0.44 0.44 0.67 0.67 $1 $0 $0 $1 B B D D 0.2 0.2 0.44 0.44 0.5 0.5 0.8 0.8 -$0.04 +$0.76 0.5 0.5 0.12 0.12 -$0.022 -$0.022 C C Example • Solution: xA=0.6 xB=0.8 xC=0.7 xD=0.9

13. Average $0 -$0.284 $0 -$0.222 $0 -$0.222 -$0.422 -$0.2 $0.8 $0.578 $0.382 $0.444 $0.738 -$0.618 -$0.556 -$0.262 $0 $0 $0 -$0.516 -$0.222 -$0.222 -$0.578 -$0.262 -$0.2 -$0.2 -$0.494 -$0.422 -$0.2 -$0.556 -$0.2 -$0.484 -$0.2 -$0.506 -$0.222 -$0.444 $0.728 -$0.272 0.6 0.33 A 0.44 0.67 B D 0.2 0.44 0.5 0.8 0.5 0.12 C Play the game $1 Pocket: -$0.2 $1 $1 -$0.05 -$0.04 -$0.022 • Walk results: $0.578 $0.444 $0.8 $0.728 …… $0.738 $0.382

14. Start Start Reusing computations: avoiding repeated walks(When solving for all xi values) New home Previously calculated node Benefit: more and more homes shorter and shorter walks one walk = average of multiple walks Qian et al., DAC2003

15. Reusing computations: “Journey record”(when solving for multiple right hand sides) Keep a record: motel/award list New RHS: Ax = b2 Update motel prices, award values Use the record: pay motels, receive awards New solution Benefit: no more walks only feasible after trick#1 Qian et al., DAC2003

16. Error vs. runtime tradeoffs • Industrial circuit • 70729 nodes, 31501 bottom-layer nodes • VDD net true voltage range 1.1324—1.1917

17. Random walk overview • Advantage • Locality: solve single entry • Weakness • Error ~ M-0.5 • 3% error to be faster than direct/iterative solvers

18. Example application: Power grid analysis • Exact DC analysis: Solve GX = E • Very expensive to solve for millions of nodes, eventually prohibitive • Simple observations: • VDD and GND pins all over chip surface (C4 connections) • Most current drawn from “nearby” connections

19. A preconditioned iterative solver

20. Current techniques • Preconditioning • Popular choice: Incomplete LU • Placement/thermal matrices • Symmetric positive definite • Popular choice: ICCG with • Different ordering • Different dropping rules

21. Current techniques done done • Rules • Pattern • Min value • Size limit

22. stochastic precondi-tioning adopt an iterative scheme Iterative Solver The Hybrid Solver Stochastic Solver Stochastic preconditioning • Special case today • Symmetric • Positive diagonal entries • Negative off-diagonal entries • Irreducibly diagonally dominant • These are sufficient, NOT necessary, conditions

23. Stochastic Solver New solution: Sequential Monte Carlo Stochastic Solver approx. solve error & residual approx. solve Benefit: ||r||2<<||b||2 ||y||2<<||x||2 same relative error = lower absolute error A. W. Marshall 1956, J. H. Halton 1962

24. Keep a record: motel/award list Stochastic Solver New RHS: Ax = b2 Update motel prices, award values Stochastic Solver Use the record: pay motels, receive awards New solution New solution: This is computationally easy! Can show that this amounts to preconditioned Gauss-Jacobi So - why G-J? Why not CG/BiCG/MINRES/GMRES

25. Solver using the record What’s on the journey record? Row i These are “UL” factors. Relation to LU factors? Just a matrix reordering!

26. Recall LDL factorization • What we need for symmetric A • What we have • How to find ? • Easy – details omitted here • Easy extension to asymmetric matrices exists

27. Compare to existing ILU • Existing ILU • Gaussian elimination • Drop edges by pattern, value, size • Error propagation • A missing edge affects subsequent computation • Exacerbated for larger and denser matrices b2 b2 b1 b1 a b3 b3 b5 b5 b4 b4

28. Superior because … • Each row of L is independently calculated • No knowledge of other rows • Responsible for its own accuracy • No debt from other steps

29. Test setup • Quadratic placement instances • Set #1: matrices and rhs’s by an industrial placer • Set #2: matrices by UWaterloo placer on ISPD02 benchmarks, unit rhs’s • LASPACK: ICCG with ILU(0) • MATLAB: ICCG with ILUT • Approx. Min. Degree ordering • Tuned to similar factorization size • Same accuracy: • Complexity metric: # double-precision multiplications • Solving stage only

30. Comparison

31. Physical runtimes on P4-2.8GHz • Reasonable preconditioning overhead • Less than 3X solving time • One-time cost, amortized over multiple solves

32. Scaling trend

33. Newer results • Examples generated from Sparskit • Finite difference discretization of 3D Laplace’s equation under Dirichlet boundary conditions • S: #nonzeros in preconditioner (similar in all cases) • C1: Condition # of original matrix • C2: Condition # after split preconditioning Ex1 Ex2 Ex3 Ex4 Ex5 Ex6

34. Conclusion • Direct solver • Locality property • Reasonable for approximate solutions • Hybrid solver • Stochastically preconditioned iterative solver • Independent row/column estimates for LU factors • Better quality than same-size traditional ILU • Extension to non-diagonally dominant matrices? • Some pointers exist (see Haifeng Qian’s thesis) • Needs further work…

35. Thank you! • Downloads • Solver package http://www.ece.umn.edu/users/qianhf/hybridsolver • Thesis http://www-mount.ece.umn.edu/~sachin/Theses/HaifengQian.pdf

36. Thanks also (and especially) to… Haifeng Qian (He will pick up the 2006 ACM Oustanding Dissertation Award in Electronic Design Automation in San Diego next week)