MA2213 Lecture 5

1. MA2213 Lecture 5 (Direct Solvers) Linear Equations

2. Systems of Linear Equations p. 243-248 Occur in a wide variety of disciplines Anthropology Astronomy Archaeology Biology Business Chemistry Economics Engineering Finance Geology Mathematics Management Medicine Physics Psychology Sociology Statistics

3. Matrix Form for a system of linear equations coefficient matrix (right) column vector (solution) column vector

4. Linear Equations in Mathematics Numerical Analysis Coefficient Matrix Interpolation Vandermonde (for polyn. interp.) Least Squares or Gramm Transpose of Vandermonde Lec 4 vufoil 13 (to compute weights) Quadrature Geometry find intersection of lines or planes Algebra partial fractions

5. Matrix Arithmetic p. 248-264 Matrix Multiplication Identity Matrix Matrix Inverse Theorem 6.2.6 p. 255 A square matrix has an inverse iff (if and only if) its determinant is not equal to zero.

6. Solution of for nonsingular (this means exists and is unique. Proof multiplication is associative Remark In MATLAB use: x = A \ b;

7. Column Rank of a Matrix Definition The column rank of a matrix is the dimension of the subspace of spanned by the column vectors of Remark maximal numberof linearly independent column vectors of Question

8. Row Rank of a Matrix Definition The row rank of a matrix is the dimension of the subspace of spanned by the row vectors of Remark maximal numberof linearly independent row vectors of Question

9. A Matrix Times a Vector The equation has solution iff b is a linear combination of columns of A

10. Existence of Solution in General The linear equation called Augmented matrix p. 265 has a solution if and only if Example EVEN IF IS SINGULAR! this has a solution iff then it has an infinite number of solutions

11. Computing the Column and Row Ranks The ranks of a matrix can be computed using a sequence of elementary row operations p. 253-254. i. Interchange two rows ii. Multiply a row by a nonzero scalar iii. Add a nonzero multiple of one row to another row Question Show that each of the ERO i, ii, iii has an inverse ERO i, ii, iii.

12. Elementary Row Operations on a matrix can be performed by multiplying on the left by nonsingularmatrices

13. Invariance of Row Rank Under ERO Theorem 1. If and is an ERO matrix, then Proof Clearly, interchanging two rows and multiplying a row by a nonzero scalar does not change the row rank. Finish the proof by showing that adding a multiple of any row to another row does not change the row rank. Remark Clearly the row rank of a matrix is invariant under sequence of ERO’s.

14. Matrix Multiplication

15. Invariance of Column Rank under ERO Theorem 2 If and is nonsingular then Proof It suffices to show that for a set of column vectors of are linearly dependent iff the set of column vectors of are linearly dependent. Show why it suffices and then show it. Hint: prove

16. Row Echelon Matrices Definition A matrix is called an row echelon matrix if i. the nonzero rows come first ii. the first nonzero element in each row =1 (called a pivot) has all zeros below it iii. each pivot lies to the right of the pivot in the row above

17. Row Echelon Matrices These three properties produce a staircase patternin the matrix below Question Where are the pivots ?

18. Row Rank of an Row Echelon Matrix equals the number of nonzero rows. Question What is the rank of this matrix ? Prove this by showing that the rows must be linearly independent. Hint : use pivots.

19. Col. Rank of a Row Echelon Matrix equals the number of nonzero rows. Question Show this by showing that the col. vectors that contain pivots form a basis for the space spanned by col. vectors. Hint: do elem. col. operations on the matrix above.

20. Reduction to Row Echelon Form Theorem 3 For every matrix there exists a nonsingular matrix such that is an echelon matrix. Furthermore, the matrix is a product where each is an ERO matrix. Application of the sequence of ERO’s is called reduction to row echelon form. Proof Based on Gaussian elimination.

21. Row Rank = Column Rank Theorem 4 For every matrix Proof. Theorem 3 implies that there exists a product of ERO matrices such that is a row echelon matrix. Theorems 1 implies that and theorems 2 implies that Since is a row echelon hence matrix,

22. Applications of Row Echelon Reduction has a solution The linear equation iff the last nonzero row of the reduced has its pivot NOT in the last column. Example Hence the condition above is satisfied iff

23. Applications of Row Echelon Reduction A basis of column vectors for a matrix can be obtained by first computing the reduction then choosing the column vectors that contain the pivots. Then the vectors are column vectors of that form a basis for the space spanned by the column vectors of

24. Generalities on Gaussian Elimination Gaussian elimination is the process of reducing a matrix to row echelon form through a sequence of ERO’s. It can also be used to solve a system of linear equations It is ‘best’ taught through showing examples. We will show how to solve a system of linear equations using Gaussian elimination, it will become obvious how to use Gaussian elimination for reduction. The final step of solving a system of equations after the augmented matrix has been reduced is called back substitution, this process is related to elementary column operations and will be addressed in the homework.

25. Gaussian Elimination (p. 264-269) Case 1. Question What type of matrix is this ? The equations for this matrix are therefore, if A is nonsingular then Question How do we use the nonsingular assumption?

26. Back Substitution Question What is this matrix called ? Case 2. Question What are the associated equations ? A nonsingular  solution by back-substitution p. 265 Question How do we use the nonsingular assumption? Question Why is this method called back-substitution ?

27. Gaussian Elimination on Equations Case 3. Apply elementary row operations on equations to to obtain equations with an upper triangular matrix Question How can we solve these equations ?

28. Gaussian Elimination on Augmented Matrix

29. Gaussian Elimination Question What is the solution ?

30. Partial Pivoting p. 270-273 For the j-th column in Gaussian elimination compute that gives the integer then perform the row interchange Read p. 273-276 about how Gaussian elimination can be used to compute the inverse of a matrix.

31. LU Decomposition p. 283-285 with same To solve for many values of first compute the factorization where Then for each b use forward substitution to solve L y = b then use backward substitution tosolve U x = y.

32. LU Decomposition Algorithm Algorithm Step 1 Step 2 for r = 2,…,n do Question How many operations does this require ?

33. Homework Due Tutorial 3 Question 2. Prove that the column rank of an row echelon matrix equals the number of nonzero rows by showing that the set of its column vectors having pivots is a maximal set of linearly independent column vectors. Question 1. Prove that the row rank of an row echelon matrix equals the number of nonzero rows. Question 3. Use Gaussian elimination to solve Question 4. Derive expressions for the entries of the L and U in the LU decomposition of a 3 x 3 matrix A. Question 5. Show how elementary column operations can be applied to a row echelon matrix M to obtain a row echelon matrix with exactly one 1 in each nonzero row. Use this to determine a basis for the space { x : Mx = 0 }.