Canonical Correlation Analysis

1. Canonical Correlation Analysis Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking and Multimedia

2. Canonical Correlation Analysis • Seeks to identify and quantify the association between two sets of variables • Examples • Relating arithmetic speed and arithmetic power to reading speed and reading power • Relating government policy variables with economic goal variables • Relating college “performance” variables with precollege “achievement” variables

3. Canonical Correlation Analysis • Focuses on the correlation between a linear combination of the variables in one set and a linear combination of the variables in another set • First to determine the pair of linear combinations having the largest correlation • Next to determine the pair of linear combinations having the largest correlation among all pairs uncorrelated with the initially selected pair, and so on

4. Canonical Correlation Analysis • Canonical variables • Pairs of linear combinations used in canonical correlation analysis • Canonical correlations • Correlations between the canonical variables • Measures the strength of association between the two sets of variables • Maximization aspect • Attempt to concentrate a high-dimensional relationship between two sets of variables into a few pairs of canonical variables

5. Example 10.5 Job Satisfaction

6. Example 10.5 Job Satisfaction

7. Canonical Variables and Canonical Correlations

8. Canonical Variables and Canonical Correlations • Covariances between pairs of variables from different sets are contained in S12 or, equivalently S21 • When p and q are relatively large, interpreting the elements of S12 collectively is very difficult • Canonical correlation analysis can summarize the associations between two sets in terms of a few carefully chosen covariances rather than the pq covariances in S12

9. Canonical Variables and Canonical Correlations

10. Canonical Variables and Canonical Correlations • First pair of canonical variables • Pair of linear combinations U1, V1 having unit variances, which maximize the correlation • kth pair of canonical variables • Pair of linear combinations Uk, Vk having unit variances having unit variances, which maximize the correlation among all choices uncorrelated with the previous k-1 canonical variable pairs

11. Result 10.1

12. Result 10.1

13. Result 10.1

14. Proof of Result 10.1

15. Proof of Result 10.1

16. Proof of Result 10.1

17. Proof of Result 10.1

18. Proof of Result 10.1

19. Canonical Variates

20. Comment

21. Comment

22. Example 10.1

23. Example 10.1

24. Example 10.1

25. Alternative Approach

26. Identifying Canonical Variablesby Correlation

27. Example 10.2

28. Canonical Correlations vs. Other Correlation Coefficients

29. Example 10.3

30. Sample Canonical Variates and Sample Canonical Correlations

31. Result 10.2

32. Matrix Forms

33. Sample Canonical Variates for Standardized Observations

34. Example 10.4

35. Example 10.5 Job Satisfaction

36. Example 10.5 Job Satisfaction

37. Example 10.5: Sample Correlation Matrix Based on 784 Responses

38. Example 10.5: Canonical Variate Coefficients

39. Example 10.5: Sample Correlations between Original and Canonical Variables

40. Matrices of Errors of Approximations

41. Matrices of Errors of Approximations

42. Matrices of Errors of Approximations

43. Example 10.6

44. Example 10.6

45. Example 10.6

46. Sample Correlation Matrices between Canonical and Component Variables

47. Proportion of Sample Variances Explained by the Canonical Variables

48. Proportion of Sample Variances Explained by the Canonical Variables

49. Example 10.7

50. Result 10.3