Understanding ANOVA: Analysis of Variance for Sample Means and their Differences

1. First There Was the t-Test the Psych Dept was Safe Just When You Thought Then Came ANOVA!

2. ANOVA • Analysis of Variance: • Why do these Sample Means differ as much as they do (Variance)? • Standard Error of the Mean (“variance” of means) depends upon • Population Variance (/n) • Why do subjects differ as much as they do from one another? • Many Random causes (“Error Variance”) • or • Many Random causes plus a Specific Cause (“Treatment”) Making Sample Means More Different than SEM

3. Why Not the t-Test • If 15 samples are ALL drawn from the Same Populations: • 105 possible comparisons • Expect 5 Alpha errors (if using p<0.05 criterion) • If you make your criterion 105 X more conservative • (p<0.0005) you will lose Power

4. The F-Test • ANOVA tests the Null hypothesis that ALL Samples came from • The Same Population • Maintains Experiment Wide Alpha at p<0.05 • Without losing Power • A significant F-test indicates that At Least One Sample • Came from a different population • (At least one X-Bar is estimating a Different Mu)

5. The Structure of the F-Ratio Estimation (of SEM) The Differences (among the sample means) you got ---------------------------------------------------------------- The Differences you could expect to find (If H0 True) Expectation F = Evaluation (If this doesn’t sound familiar, Bite Me!)

6. The Structure of the F-Ratio If H0 True: Average Error of Estimation of Mu by the X-Bars ---------------------------------------------------------------- Variability of Subjects within each Sample F = • Size of Denominator determines size of Numerator • If a treatment effect (H0 False): • Numerator will be larger than predicted by • denominator

7. The Structure of the F-Ratio Between Group Variance ------------------------------- Within Group Variance F = If H0 True: Error Variance ------------------ Error Variance Approximately Equal With random variation F = If a treatment effect (H0 False): Error plus Treatment Variance ------------------------------------- Error Variance Numerator is Larger F =

8. Probability of F  as F Exceeds 1 Between Group Variance ------------------------------- Within Group Variance F = If H0 True: Error Variance ------------------ Error Variance Approximately Equal With random variation F = If a treatment effect (H0 False): Error plus Treatment Variance ------------------------------------- Error Variance Numerator is Larger F =

9. For U Visual Learners Sampling Distributions H0 True: H0 False: Reflects SEM (Error) Error Plus Treatment

10. Keep the Data, Burn the Formulas

11. Do These Measures Depend on What Drug You Took? • Drug A & B don’t look different, but Drug C looks different • From Drug A & B

12. Partitioning the Variance • Each Subject’s deviation score can be decomposed into 2 parts: • How much his Group Mean differs from the Grand Mean • How he differs from his Group Mean • If Grand Mean = 100: • Score-1 in Group A =117; Group A mean =115 • (117 - 100) = (115 - 100) + (117 - 115) • 17 = 15 + 2 • Score-2 in Group A = 113; Group A mean = 115 • (113 – 100) = (115 - 100 + (113 – 115) • 13 = 15 - 2

13. Partitioning the Variance in the Data Set • Total Variance (Total Sum of Squared Deviations from Grand Mean) • Sum (Xi-Grand Mean)^2 Variance among Samples Sum (X-Bar – Grand Mean)^2 For all Sample Means Variance among Subjects Within each group (sample) Sum ( Xi – Group mean)^2 for All subjects in all Groups SS-Total SS-Between SS-Within

14. Step 1: Calculate SS-Total

15. Step 2: Calculate SS-Between • Multiply by n (sample size) because: • Each subject’s raw score is composed of: • A deviation of his sample mean from the grand mean • (and a deviation of his raw score from his sample mean)

16. Step 3: Calculate SS-Within SS-Total – SSb = SSw 84.91667 – 60.6667 = 24.25 Should Agree with Direct Calculation

17. Direct Calculation of SSw

18. Step 4: Use SS to ComputeMean Squares & F-ratio • The differences among the sample means are over 11 x greater than if: • All three samples came from the Same population • None of the drugs had a different effect • Look up the Probability of F with 2 & 9 dfs • Critical F2,9 for p<0.01 = 8.02 • Reject H0 • Not ALL of the drugs have the same effect

19. The F-Table

20. The ANOVA Summary Table

21. What Do You Do Now? • A Significant F-ratio means at least one Sample came from a • Different Population. • What Samples are different from what other Samples? • Use Tukey’s Honestly Significant Difference (HSD) Test

22. Tukey’s HSD Test • Can only be used if overall ANOVA is Significant • A “Post Hoc” Test • Used to make “Pair-Wise” comparisons • Structure: • Analogous to t-test • But uses estimated Standard Error of the Mean in the Denominator • Hence a different critical value (HSD) table

23. Tukey’s HSD Test Unequal N Equal N

24. Assumptions of ANOVA • All Populations Normally distributed • Homogeneity of Variance • Random Assignment • ANOVA is robust to all but gross violations of these theoretical • assumptions

25. Effect Size S = 0.10 M = 0.25 L = 0.40 MStreatment is really MSb Which is T + E What’s the Question?