On implicit-factorization block preconditioners

1. On implicit-factorization block preconditioners Sue Dollar1,2, Nick Gould3, Wil Schilders2,4 and Andy Wathen1 1 Oxford University Computing Laboratory, Oxford, UK 2 CASA, Technische Universiteit Eindhoven, The Netherlands 3 Rutherford Appleton Laboratory, Chilton, UK 4 Philips Research Laboratory, Eindhoven, The Netherlands

2. Summary of the talk • Introduction • Direct versus Iterative Methods • Preconditioned Conjugate Gradient Method • Constraint Preconditioners • Implicit Constraint Preconditioners • Numerical Examples • Future work and conclusions

3. Introduction Interested in solving structured linear systems of equations • full rank • positive semi-definite

4. Example 1: Equality constrained minimization Interior-point sub-problem • Θ (small) barrier weights First-order optimality

5. Factory

6. Example 2: Inequality constrained minimization Interior-point sub-problem • C (small) barrier weights First-order optimality

7. Direct vs. Iterative Methods • Direct methods • Gaussian elimination with a pivoting strategy • Bunch-Parlett factorization • Iterative methods • Krylov subspace methods • MINRES & GMRES – find solution of (1) within n+m iterations • CG may fail because of the indefiniteness of system • Often advantageous to use a preconditioner P

8. PCG Possible to use the preconditioned conjugate-gradient method (2) (Gould, Hribar, Nocedal)

9. Projected PCG • Can perform iteration in the original (x, z) space so long as preconditioner chosen carefully … Solve Iterate until convergence • … • Solve • …

10. Constraint Preconditioners Case C=0 • The matrix P-1H has • an eigenvalue at 1 with multiplicity 2m • n-m eigenvalues which are defined by the generalized eigenvalue problem (Keller, Gould, Wathen)

11. Case rank(C)=l • The matrix P-1H has • at least 2m-l eigenvalues at 1 • n-m eigenvalues which are defined by the generalized eigenvalue problem • m eigenvalues defined by the generalized eigenvalue problem • where w=[xy;xz], subject to Y2BY1xy≠0.

12. CVXQP1_S: m=50, n=100, G=diag(A)

13. CVXQP1_S: m=50, n=100, G=diag(A)

14. Explicit vs. implicit constraint preconditioner Explicit constraint preconditioners: choose G, and then factorize Implicit constraint preconditioners: find easily-invertible factors R and S so that always holds Pick parts of R and S to match * (≡ G) to parts of A

15. Easily invertible B1 • Require that both R and S are easily block invertible • - some sub-blocks should be zero

16. Example 1: C=0 • can recover any G(Schilders)

17. Example 2 Example 3

18. Numerical Examples CUTEr test set Case C=0 26/40 problems solved faster (20 if take into account perm time)

19. Case C=I 29/40 problems solved faster (25 if take into account perm time)

20. Conclusions • New method for constructing preconditioners for CG methods for a variety of important problems • Preconditioners • implicitly respect crucial structure • easy to apply • flexible and capable of improving eigenvalue clusters • Extend the class of problems for which CG is applicable • Still under development…but will be available as part of GALAHAD Open questions • How to pick basis B1 efficiently and so as to improve eigenvalue clusters • How to approximate blocks of H in G • What about Mr Greedy?

22. Conclusions • New method for constructing preconditioners for CG methods for a variety of important problems • Preconditioners • implicitly respect crucial structure • easy to apply • flexible and capable of improving eigenvalue clusters • Extend the class of problems for which CG is applicable • Still under development…but will be available as part of GALAHAD Open questions • How to pick basis B1 efficiently and so as to improve eigenvalue clusters • How to approximate blocks of H in G