Numerical Analysis

1. Numerical Analysis EE, NCKU Tien-Hao Chang (Darby Chang)

2. In the previous slide • Error estimation in system of equations • vector/matrix norms • LU decomposition • split a matrix into the product of a lower and a upper triangular matrices • efficient in dealing with a lots of right-hand-side vectors • Direct factorization • as an systems of n2+n equations • Crout decomposition • Dollittle decomposition

3. In this slide • Special matrices • Strictly diagonally dominant matrix • Symmetric positive definite matrix • Cholesky decomposition • Tridiagonal matrix • Iterative techniques • Jacobi, Gauss-Seidel and SOR methods • conjugate gradient method • Nonlinear systems of equations • Exercise 3

4. 3.7 Special Matrices

5. Special matrices • Linear systems • which arise in practice and/or in numerical methods • the coefficient matrices often have special properties or structure • Strictly diagonally dominant matrix • Symmetric positive definite matrix • Tridiagonal matrix

6. Strictly diagonally dominant

8. Symmetric positive definite

9. Symmetric positive definiteTheorems for verification

11. Symmetric positive definiteRelations to • Eigenvalues • Leading principal sub-matrix

12. Cholesky decomposition • For symmetric positive definite matrices • greater efficiency can be obtained • consider the symmetric of the matrix • Rather than LU form, we factor the matrix into the form • A=LLT

15. Tridiagonal • Only 8n-7 operations • factor step 3n-3 • solve step 5n-4

17. 3.7 Special Matrices

18. Before entering 3.8 • So far, we have learnt three methods algorithms in Chapter 3 • Gaussian elimination • LU decomposition • direct factorization • Are they algorithms? • What’s the differences to those algorithms in Chapter 2? • they report exact solutions rather than approximate solutions question further question answer

19. 3.8 Iterative Techniques for Linear Systems

20. Iterative techniques • Analytic techniques is slow • O(n3) • Especially for systems with large but sparse coefficient matrices • As an added bonus, iterative techniques are less insensitive to roundoff error

21. Iterative techniquesBasic idea

22. Iteration matrixImmediate questions • When does T guarantee a unique solution? • When does T guarantee convergence? • How quick does {x(k)} converge? • How to generate T?

23. Assume that I-T is singular, there exists a nonzero vector x such that (T-1I)x=0 • 1 is a eigenvalue of T • but ρ(T)<1, contradiction

26. (in section 2.3 with proof) Recall that http://www.dianadepasquale.com/ThinkingMonkey.jpg

27. Recall that http://www.dianadepasquale.com/ThinkingMonkey.jpg

28. Iteration matrixFor these questions • We know that when ρ(T)<1, {x(k)} from x(k+1)=Tx(k)+c will converge linearly to a unique solution with any initial vector x(0) • What is missing? • remember the problem is to solve Ax=b • How to generate T? • like f(x) and g(x), different algorithms construct different iteration matrix question hint answer

29. Splitting Methods

30. Splitting methods • Ax=b (M–N)x=b  Mx=Nx+b • x=M-1Nx+M-1b • T=M-1N and c=M-1b • A class of iteration methods • Jacobi method • Gauss-Seidel method • SOR method

33. Gauss-Seidel method

34. Gauss-Seidel methodIteration matrix

35. The SOR method (successive overrelaxatoin)

36. Iterative Techniques for Linear Systems

37. 3.9 Conjugate Gradient Method 37

38. Conjugate gradient method • Not all iterative methods are based on the splitting concept • The minimization of an associated quadratic functional

39. Conjugate gradient methodQuadratic functional

40. http://fuzzy.cs.uni-magdeburg.de/~borgelt/doc/somd/parabola.gifhttp://fuzzy.cs.uni-magdeburg.de/~borgelt/doc/somd/parabola.gif

42. Minimizing quadratic functional

43. Choose the search direction d(m) • as the tangent line in Newton’s method • the gradient of f at x(m) • Choose the step size • as the root of the tangent line

44. http://www.mathworks.com/cmsimages/op_main_wl_3250.jpg Global optimization problem

45. Conjugate Gradient Method

46. 3.10 Nonlinear Systems of Equations

47. Nonlinear systems of equations

48. Generalization of root-finding

49. GeneralizationNewton’s method

50. Generalization of Newton’s methodJacobian matrix