BIOL 582

1. BIOL 582 Supplemental Material Matrices, Matrix calculations, GLM using matrix algebra

2. BIOL 582 Why bother with matrices? • Compact method of expressing mathematical operations (including statistics) • Generalize from one to many variables (i.e. vectors to matrices) • Matrix operations have geometric interpretations in data spaces • Many data types (e.g., morphometric data) cannot be measured with a single variable, so multivariate methods are required to properly address hypotheses

3. BIOL 582 Scalars, vectors, and matrices • Scalar: a number • Vector: an ordered list (array) of scalars (nrows x 1cols) • Matrix: a rectangular array of scalars (nrows x pcols)

4. BIOL 582 Matrix transpose • Reverse rows and columns • Represent by At or A′ • Vector transpose works identically

5. BIOL 582 Matrix addition and subtraction • Matrices must have same dimensions • Add/subtract element-wise • Vector addition/subtraction works identically Addition Subtraction

6. BIOL 582 Matrix multiplication • Scalar multiplication: Multiply scalar by each element in matrix or vector • Matrix/vector multiplication is a summed multiplication • Inner dimensions allow multiplication • Outer dimensions determine size of result • Order of matrices makes a difference: AB BA AB n1× p1 * n2 × p2 inner

7. BIOL 582 Matrix multiplication • Scalar multiplication: • Matrix multiplication: Inner dimensions MUST AGREE!!!

8. BIOL 582 Matrix multiplication: inner and outer products • Inner (scalar) product: vector multiplication resulting in a scalar (weighted linear combination) • Outer (matrix) product: vector multiplication resulting in a matrix Inner Product Outer Product Inner dimensions MUST AGREE!!!

9. BIOL 582 Special matrices • I: Identity matrix (equivalent to ‘1’ for matrices) • 1: A matrix of all ones • 0: A matrix of all zeros • Diagonal: diagonal contains non-zero elements • Square: n = p • Symmetric: off-diagonal elements same:

10. BIOL 582 Special matrices

11. BIOL 582 Special matrices • Orthogonal: square matrix with property: • VERY useful for statistics and other fields (e.g, morphometrics) • Orthogonal matrices can be thought of as rigid rotations of data sets (shown later) Orthonormal Example:

12. BIOL 582 Matrix inversion • Can’t divide matrices , so calculate the inverse (reciprocal) of denominator and multiply • Inverses have property that: • Inverses are tedious to calculate, so in practice we use a computer • Only works for square matrices whose determinant 0!!! • Determinant: combination of diagonal and off-diagonal elements • A matrix whose determinant = 0 is Singular (has no inverse) For:

13. BIOL 582 Matrix inversion: example • For the 2 x 2 case: • Example: • Confirm:

14. BIOL 582 Matrix multiplication: geometric interpretations • Multiplying data and other matrices has geometric interpretations • XI=X: No change to X • cIX=Y: Change of scale (e.g, enlargement) • XD=Y: Stretching if D is diagonal • XT=Y: Rigid rotation if T is p×p orthogonal • XT=Y: Shear if T is not orthogonal • (T can be decomposed into rotation, dilation, rotation) • X = data matrix

15. BIOL 582 Matrix multiplication: visual examples Scalar (1/2) Original Scalar (2) Rotations Shears and Projections (images from C.A.B. Smith, 1969)

16. BIOL 582 GLM in matrix form Note, in general vectors are lower case and matrices are upper case, but using upper case is more encompassing • The GLM model: • Independent Variable/s: • Dependent Variable/s: • Solve for ‘regression coefficients’ • b found from:

17. BIOL 582 GLM in matrix form: Solving for β This is the model • Why this equation? Start with: Make ‘X’ a square matrix: Multiply by inverse:

18. BIOL 582 GLM in matrix form: Deriving univariate regression 1. Expand matrixes: where: and: 2. Begin rewrite:

19. BIOL 582 GLM in matrix form: Deriving univariate regression 2. From before: 3. Calculate inverse: 4. Multiply

20. BIOL 582 GLM in matrix form: Calculating sums of squares (SS) • F-ratio is: SSM/SSE (with df corrections) • Need to calculate full and reduced model SS • Full model (contains all terms) • Reduced model (X# has 1 less term – column of x values – in it) • Significance based on: • Or one can always use a random permutation approach…

21. BIOL 582 Regression example • The Data: (for matrix form):

22. BIOL 582 Using GLM for ANOVA • Analysis of Variance (ANOVA) is the standard way of comparing means among multiple groups. • ANOVA is the cornerstone of most applied stats courses in life science fields • Linear regression equation • ANOVA equation

23. BIOL 582 Using GLM for ANOVA • Same idea, but must use special X-matrix coding • Recode k groups in k-1 dummy variables columns) of X • Generally, column 1 yields , column 2 yields deviation from for mean of group 1, etc.

24. BIOL 582 Using GLM for ANOVA: Example 1 • The Data: n1=5 n2=5 (Reduced model is one column of 1s) DEMO

25. BIOL 582 GLM final comments • As we will learn, ANOVA, ANCOVA, Multiple Regression, MANOVA, MANCOVA, and Multivariate Multiple Regression, are all variants of the same GLM procedure. • All of these “different” analytical approaches are no different to a computer using matrix calculations to perform GLM • If there are 4 groups, then 4 – 1 = 3 dummy variables are needed. If there are 88 groups, then 88 – 1 = 87 dummy variables are needed. ALWAYS, there are a -1 “factor” levels for a groups.

26. BIOL 582 GLM final comments • Dummy variables are “indicator” variables • E.g., can be written as • where Z is an indicator: 1 if in the group; 0 if not in the group. • There are two ways to form the design matrix (X): Groups 2-4 means are expressed as deviations from the first group mean All group means are expressed as deviations from the overall group mean Analytically, these are no different, but different software packages use different approaches!