Generalized Inverses of Rectangular and Singular Square Matrices

1. Generalized Inverses of Rectangular and Singular Square Matrices Philip C. Nelson William Paterson University Department of Mathematics Advisor: Dr. Phadia Spring 2008

2. Outline • Background and Motivation • Definition of Generalized Inverse • Properties of Generalized Inverses and how they are similar to those of nonsingular matrices • Applications of Generalized Inverses • Linear Equations with m-equations and n-unknowns • where • Least Squares Method

3. Background In an undergraduate Linear Algebra course, one learns that if a matrix A is non-singular, then there exists an inverse denoted by A-1 such that AA-1 = A-1A = I, where I is the identity matrix. These inverses are unique and can be obtained through a variety of elementary methods, one of which involves performing a series of row operations.

4. Question What can be said about the inverses of rectangular or square, but singular matrices? Elementary methods are no longer applicable, so how do we define inverses for such matrices?

5. Are Rectangular Matrices Invertible? Let’s consider rectangular matrix, A, with dimensions m x n, where the rank of A is m. If we multiply A with its transpose AT, we would obtain an m x m square matrix. Should the inverse denoted by, (AAT)-1, exist, then we have the following equation: I = (AAT)(AAT)-1 By associativity (AAT)(AAT)-1 = A[AT (AAT)-1] Let AT(AAT)-1 = A We obtain I = AA . Thus A when postmultiplied to A is considered the right inverse of A.

6. Are Rectangular Matrices Invertible? Similarly, a rectangular matrix would have a left inverse A if the rank of A is n which satisfies the following equation. A A = I Only in this case, A = (ATA)-1AT. Note that either a right or left inverse will exist only if the rank of A is either nor m respectively. Remark with nonsingular matrices: The rank of A in such case is m = n. Using the ideas of Left and Right Inverses, it can concluded that they are the same hence why they are unique and commutable.

7. Generalized InverseDefinition Recall Linear Equations of the form Ax = y If A is nonsingular, then we define the solution space through x = A-1y Suppose A is rectangular or singular-square, how would we solve for x? If there exists a matrix G such that x = Gy is a solution of Ax = y, then it can be said that G behaves as the inverse of A, Hence G is called a generalized inverse of A. (Important note: G need not be unique)

8. Generalized InverseExistence G exists if and only if AGA = A. • Remark with nonsingular matrices: • For a nonsingular matrix A of dimension n, • We say the inverse of A exists if and only if • AA-1= I and I = A-1A if and only if • rank(A) = n if and only if • Row Operations can be applied to A to obtain I

9. Penrose InverseDefinition AGA = A GAG =G AG is hermitian GA is hermitian

10. Generalized InverseProperties G exists if and only ifGA is idempotent and rank(GA) = trace(GA)= rank(A). and G exists if and only ifAG is idempotent and rank(AG) = trace(AG) = rank(A). Note: A matrix A is idempotent if A2 = A. Remark with nonsingular matrices: AA-1= A-1A = I I is idempotent as I2 = I rank(I) = trace(I) = rank(A)

11. Generalized InverseProperties G exists if and only if rank(G) ≥ rank(A) Remark with nonsingular matrices: A-1exists if and only if rank(A-1) = rank(A)

12. Generalized InverseProperties One choice of (AT)-1 is (A-1) T A(ATA)-1(ATA) = A and (ATA)(ATA)-1AT = AT ATAGA = ATAif and only if AGA = A

13. Generalized InverseProperties A(A*A)-1A* is hermitian and idempotent (Note: A matrix A is hermitian if A* = A.) If BC is idempotent and rank(BC) = rank(B) or rank(C), then CB is idempotent.

14. Generalized InverseUsage in Systems of Linear Equations Let A have dimensions m x n and G be any g-inverse of A. (Further let v be an arbitrary vector.) Then the following hold: A general solution of the homogeneous equation Ax = 0 is x = (I – GA)v A general solution of a consistent nonhomogeneous equation Ax = y is x = Gy + (I – GA)v A necessary and sufficient condition that Ax = y is consistent is that AGy = y. • Remark with nonsingular matrices: • Suppose G = A-1, then (I-GA) would be 0, thus eliminating the use of the arbitrary v vector.

15. ApplicationsSolutions to Systems of Equations

16. ApplicationsSolutions to Systems of Equations

17. ApplicationsSolutions to Systems of Equations

18. ApplicationsSolutions to Systems of Equations

19. ApplicationsSolutions to Systems of Equations

20. ApplicationsSolutions to Systems of Equations Recall before we claim G is a generalized inverse of A, we need to check to see if AGA = A.

21. ApplicationsSolutions to Systems of Equations Recall A general solution of a consistent nonhomogeneous equation Ax = y is x = Gy + (I – GA)v

22. ApplicationsSolutions to Systems of Equations Thus the solution space for x, while dependent on arbitrary vector v is as follows:

23. ApplicationsSolutions to Systems of Equations When we plug back into Ax = y to verify:

24. ApplicationsSolutions to Systems of Equations Remember we claimed that G was not unique. Recall our result for F-: If we change the entries in the last row to any real or complex number, we will obtain different g-inverses

25. ApplicationsSolutions to Systems of Equations Is AGA = A?

26. ApplicationsSolutions to Systems of Equations Ax = y is x = Gy + (I – GA)v

27. ApplicationsSolutions to Systems of Equations Thus the solution space for x, while dependent on arbitrary vector v is as follows:

28. ApplicationsSolutions to Systems of Equations When we plug back into Ax = y to verify:

29. Wait! Why bother with this method? For the system: And then by substitution obtain: Finding the solution to the system this way however gives no indication as to what G can be. So why bother with G anyway?

30. ApplicationsLeast Squares Approximation

31. Additional Applications • Magnetic Fields • Oceanography • Planetary Magnetic Fields • Electronics • Parallel Sum of Resistors • Computing Input and Output for Robotic Motions

32. References • Ben-Israel, Adi and Greville, Thomas. Generalized Inverses: Theory and Applications. John Wiley & Sons, Inc. New York, NY: 1974. • Gill, Jeff. What is the Generalized Inverse of a Matrix? Online. http://artsci.wustl.edu/~jgill/papers/ginv.pdf • Greenberg, Harvey J. Generalized Inverse Example. Online. http://glossary.computing.society.informs.org/examples/generalized_inverse.pdf • Kolman, Bernard and Hill, David R. Elementary Linear Algebra. 7th ed. Prentice Hall. Upper Saddle River, NJ: 2000. • Rao, C. Radhakrishna and Mitra Sujit Kumar. Generalized Inverse of Matrices andits Applications. John Wiley & Sons, Inc. New York, NY: 1971.