Understanding Correlation and Regression: Key Concepts and Formulas

1. Overview • Correlation • -Definition • -Deviation Score Formula, Z score formula • -Hypothesis Test • Regression • Intercept and Slope • Unstandardized Regression Line • Standardized Regression Line • Hypothesis Tests

2. Associations among Continuous Variables

3. 3 characteristics of a relationship • Direction • Positive(+) • Negative (-) • Degree of association • Between –1 and 1 • Absolute values signify strength • Form • Linear • Non-linear

4. Direction Positive Negative • Large values of X = large values of Y, • Small values of X = small values of Y. • - e.g. IQ and SAT • Large values of X = small values of Y • Small values of X = large values of Y • -e.g. SPEED and ACCURACY

5. Degree of association Strong(tight cloud) Weak(diffuse cloud)

6. Form Linear Non- linear

7. Regression & Correlation

8. What is the best fitting straight line? Regression Equation: Y = a + bX How closely are the points clustered around the line? Pearson’s R

9. Correlation

10. Dataset Obs X Y A 1 1 B 1 3 C 3 2 D 4 5 E 6 4 F 7 5 Correlation - Definition Correlation: a statistical technique that measures and describes the degree of linear relationship between two variables Scatterplot Y X

11. Pearson’s r A value ranging from -1.00 to 1.00 indicating the strength and direction of the linear relationship. Absolute value indicates strength +/- indicates direction

13. Cross-Product = The Logic of Correlation MEAN of X MEAN of Y For a strong positive association, the cross-products will mostly be positive

14. The Logic of Correlation MEAN of X MEAN of Y For a strong negative association, the cross-products will mostly be negative Cross-Product =

15. The Logic of Correlation MEAN of X MEAN of Y For a weak association, the cross-products will be mixed Cross-Product =

16. Deviation score formula SP (sum of products) = ∑ (X – X)(Y – Y) Pearson’s r

17. Deviation Score Formula

18. Deviation Score Formula = .99

19. Deviation score formula SP (sum of products) = ∑ (X – X)(Y – Y) Pearson’s r For a strong positive association, the SP will be a big positive number

20. ∑ (X – X)(Y – Y) Pearson’s r Deviation score formula For a strong negative association, the SP will be a big negative number SP (sum of products) =

21. ∑ (X – X)(Y – Y) Pearson’s r Deviation score formula For a weak association, the SP will be a small number (+ and – will cancel each other out) SP (sum of products) =

22. Z score formula Standardized cross-products Pearson’s r

23. Z-score formula

24. Z-score formula

25. Z-score formula r = .99

26. Formulas for R Z score formula Deviations formula

27. Interpretation of R • A measure of strength of association: how closely do the points cluster around a line? • A measure of the direction of association: is it positive or negative?

28. Interpretation of R • r = .10 very small association, not usually reliable • r = .20 small association • r = .30 typical size for personality and social studies • r = .40 moderate association • r = .60 you are a research rock star • r = .80 hmm, are you for real?

29. Interpretation of R-squared • The amount of covariation compared to the amount of total variation • “The percent of total variance that is shared variance” • E.g. “If r = .80, then X explains 64% of the variability in Y” (and vice versa)

30. Hypothesis testing with r Hypotheses H0:  = 0 HA :  ≠ 0 Test statistic = r Or just use table E.2 to find critical values of r

31. Practice

32. Practice

33. Properties of R • A standardized statistic – will not change if you change the units of X or Y. (bc based on z-scores) • The same whether X is correlated with Y or vice versa • Fairly unstable with small n • Vulnerable to outliers • Has a skewed distribution

34. Linear Regression

35. Linear Regression • But how do we describe the line? • If two variables are linearly related it is possible to develop a simple equation to predict one variable from the other • The outcome variable is designated the Y variable, and the predictor variable is designated the X variable • E.g. centigrade to Fahrenheit: F = 32 + 1.8C this formula gives a specific straight line

36. The Linear Equation • F = 32 + 1.8(C) • General form is Y = a + bX • The prediction equation: Y’ = a+ bX • Where • a = intercept • b = slope • X = the predictor • Y = the criterion a and b are constants in a given line; X and Y change

37. The Linear Equation • F = 32 + 1.8(C) • General form is Y = a + bX • The prediction equation: Y’ = a + bX • Where • a = intercept • b = slope • X = the predictor • Y = the criterion Different b’s…

38. The Linear Equation • F = 32 + 1.8(C) • General form is Y = a + bX • The prediction equation: Y’ = a + bX • Where • a = intercept • b = slope • X = the predictor • Y = the criterion Different a’s…

39. The Linear Equation • F = 32 + 1.8(C) • General form is Y = a + bX • The prediction equation: Y’ = a + bX • Where • a = intercept • b = slope • X = the predictor • Y = the criterion Different a’s and b’s …

40. Slope and Intercept • Equation of the line • The slope b: the amount of change in y with one unit change in x • The intercept a: the value of y when x is zero

41. Slope and Intercept • Equation of the line • The slope • The intercept The slope is influenced by r, but is not the same as r

42. When there is no linear association (r = 0), the regression line is horizontal. b=0. and our best estimate of age is 29.5 at all heights.

43. When the correlation is perfect (r = ± 1.00), all the points fall along a straight line with a slope

44. When there is some linear association (0<|r|<1), the regression line fits as close to the points as possible and has a slope

45. Where did this line come from? • It is a straight line which is drawn through a scatterplot, to summarize the relationship between X and Y • It is the line that minimizes the squared deviations (Y’ – Y)2 • We call these vertical deviations “residuals”

46. Regression lines Minimizing the squared vertical distances, or “residuals”

47. Unstandardized Regression Line • Equation of the line • The slope • The intercept

48. Properties of b (slope) • An unstandardized statistic – will change if you change the units of X or Y. • Depends on whether Y is regressed on X or vice versa

49. Standardized Regression Line • Equation of the line • The slope • The intercept A person 1 stdev above the mean on height would be how many stdevs above the mean on weight?

50. Properties of β (standardized slope) • A standardized statistic – will not change if you change the units of X or Y. • Is equal to r, in simple linear regression