Psych 230 Psychological Measurement and Statistics

1. Psych 230Psychological Measurement and Statistics Pedro Wolf November 18, 2009

2. This Time…. • Analysis of Variance (ANOVA) • Concepts of variability • Why bother with ANOVA? • Conducting a test

3. Statistical Testing • Decide which test to use • State the hypotheses (H0 and HA) • Calculate the obtained value • Calculate the critical value (size of ) • Make our conclusion

4. Statistical Testing • Decide which test to use • State the hypotheses (H0 and HA) • Calculate the obtained value • Calculate the critical value (size of ) • Make our conclusion • Conduct post-hoc tests

5. Analysis of Variance (ANOVA)

6. Analysis of Variance • In this statistical test, we are interested in seeing if there are significant differences between more than two groups • In an experiment involving only two conditions of the independent variable, you can use either a t-test or the ANOVA

7. Analysis of Variance • We will look at the variance within each group and compare that to the variance found between the groups • Remember: Variance is the degree to which scores are dispersed in our data

8. Analysis of Variance • Remember: H0 is that all the group means in our experiment are the same • any difference between them is due to random chance • H1 is that there is a difference between our group means • a difference that is so unlikely to have happened by chance that we conclude it is due to the independent variable

9. Analysis of Variance • There is always variability in our data • This variability can be due to two factors: • The independent variable • Systematic factors • Error variance • Random factors

10. Analysis of Variance • So, to draw a conclusion about whether the independent variable makes a difference to the dependent variable, we need to know what kind of variance there is in our data • variance due to the independent variable (systematic) • variance due to random factors (error)

11. Analysis of Variance • To assess this, we need to look at the variance both within each of our experimental conditions and also at the variance between each of our experimental conditions • Assume H0 is true - there is no effect of our independent variable • What type of variance might we expect to see?

12. Analysis of Variance • Example: we want to see if people differ in their shoe size by where they sit in the class • three groups of students: front row, middle row and back row • expect to find significant differences in shoe size? • Front row: vary in their shoe size: 6,8,5,4,7 • Middle row: vary in their shoe size : 9,9,4,6,7 • Back row: vary in their shoe size : 4,6,12,10,8

13. Analysis of Variance

14. Analysis of Variance Total variance

15. Analysis of Variance Variance within the groups Total variance

16. Analysis of Variance Variance within the groups Total variance Variance between the groups

17. Analysis of Variance Variance within the groups Total variance Variance between the groups Total Variance = Variance between + Variance within

18. Analysis of Variance • So, when H0 is actually true, we should expect the same amount of variance both within each group and between the groups • If we divide Variancebetween by Variancewithin, we should get? • If H0 is true, this ratio should be close to 1

19. Analysis of Variance • How about if H1 is actually true? • In this case, we know the independent variable is having some effect • So, we should expect more variance between each group than there is within each group

20. Analysis of Variance • Example: we want to see if people differ in their attendance by where they sit in the class • front row, middle row and back row • expect to find significant differences in attendance? • Front row: vary in their attendance: 6,5,7,6,6 • Middle row: vary in their attendance: 7,8,6,7,7 • Back row: vary in their attendance: 8,8,7,9,8

21. Analysis of Variance Variance within the groups Total variance Variance between the groups Total Variance = Variance between + Variance within

22. Analysis of Variance • When HA is true, we have more variance between each group than there is within each group • If we divide Variancebetween by Variancewithin, we should get? • If HA is true, this ratio should be more than 1 • the F-ratio

23. ANOVA • A one-way ANOVA is performed when only one independent variable is tested in the experiment • Example: we are interested in the differences between freshmen, sophomores and juniors on tests of socialization • dependent variable: socialization scores • independent variable: class standing

24. ANOVA • A two-way ANOVA is performed when two independent variables are tested in the experiment • Example: we are interested in the differences between male and female freshmen, sophomores and juniors on tests of socialization • dependent variable: socialization scores • independent variable 1: class standing • independent variable 2: gender

25. ANOVA • When an independent variable is studied using independent samples in all conditions, it is called a between-subjects factor • A between-subjects factor involves using the formulas for a between-subjects ANOVA

26. ANOVA • When a factor is studied using related (dependent) samples in all levels, it is called a within-subjects factor • This involves a set of formulas called a within-subjects ANOVA

27. ANOVA

28. ANOVA

29. ANOVA - assumptions • All conditions contain independent samples • The dependent scores are normally distributed, interval or ratio score • The variances of the populations are homogeneous

30. ANOVA - why bother? • We want to see if there are differences between our three groups: • Freshmen • Sophomores • Juniors • Why not just do a bunch of t-tests? • Freshmen vs. Sophomores • Freshmen vs. Juniors • Sophomores vs. Juniors

31. ANOVA - experiment-wise error • The overall probability of making a Type I error somewhere in an experiment is call the experiment-wise error rate • When we use a t-test to compare only two means in an experiment, the experiment-wise error rate equals a

32. ANOVA - experiment-wise error • When there are more than two means in an experiment, the multiple t-tests result in an experiment-wise error rate that is much larger than the a we have selected • Freshmen vs. Sophomores: a=0.05 • Freshmen vs. Juniors: a=0.05 • Sophomores vs. Juniors: a=0.05 • experiment-wise error rate = 0.05+0.05+0.05=approx 0.15 • Using the ANOVA allows us to compare the means from all levels of the factor and keep the experiment-wise error rate equal to a

33. Conducting an ANOVA

34. 1. Decide which test to use • Are we comparing a sample to a population? • Yes: Z-test if we know the population standard deviation • Yes: One-sample T-test if we do not know the population std dev • No: Keep looking • Are we looking for the difference between samples? • Yes: How many samples are we comparing? • Two: Use the Two-sample T-test • Are the samples independent or related? • Independent: Use Independent Samples T-test • Related: Use Related Samples T-test • More than Two: Use Anova test

35. 2. State the Hypotheses • H0 : 1 =2= ……. = k • there is no difference in the means • HA : not all sare equal • there is a difference between some of the means • Only conduct two-tailed tests using ANOVA

36. 3. Calculate the obtained value (Fobt) • The statistic for the ANOVA is F • When Fobt is significant, this indicates only that somewhere among the means at least two of them differ significantly • It does not indicate which groups differ significantly • When the F-test is significant, we perform post hoc comparisons (step 6)

37. 3. Calculate the obtained value (Fobt) • Remember, we are trying to compare the between group variance to the within group variance • We use the mean squares to calculate this • The mean square within groups is an estimate of the variability in scores as measured by differences within the conditions • The mean square between groups is an estimate of the differences in scores that occurs between the levels in a factor

38. 3. Calculate the obtained value (Fobt) • The F-ratio is therefore the mean square between groups divided by the mean square within groups

39. 3. Calculate the obtained value (Fobt) • The F-ratio is therefore the mean square between groups divided by the mean square within groups

40. 3. Calculate the obtained value (Fobt) • The F-ratio is therefore the mean square between groups divided by the mean square within groups • When H0 is true, Fobt should be close to 1 • When HA is true, Fobt should be greater than 1

41. 3. Calculate the obtained value (Fobt) • The ANOVA table:

42. 3. Calculate the obtained value (Fobt)

43. 3. Calculate the obtained value (Fobt)

44. 3. Calculate the obtained value (Fobt) dfbn= k - 1 dfwn= N - k dftot= N - 1

45. 3. Calculate the obtained value (Fobt) dfbn= k - 1 dfwn= N - k dftot= N - 1

46. Analysis of Variance dfbn= k - 1 dfwn= N - k dftot= N - 1

47. Analysis of Variance dfbn= k - 1 dfwn= N - k dftot= N - 1

48. Analysis of Variance dfbn= k - 1 dfwn= N - k dftot= N - 1

49. Analysis of Variance dfbn= k - 1 dfwn= N - k dftot= N - 1

50. Analysis of Variance dfbn= k - 1 dfwn= N - k dftot= N - 1