Statistical Analysis of Factorial Designs

1. Statistical Analysis of Factorial Designs Review of Interactions Kinds of Factorial Designs Causal Interpretability of Factorial Designs The F-tests of a Factorial ANOVA Using LSD to describe the pattern of an interaction

2. Review #1-- interaction Task Presentation Paper Computer Task Difficulty Easy 90 70 Hard 40 60 1) Put in a <, > or = to indicate the pattern of the simple effect of Task Presentation for Easy Tasks > 2) Put in a <, > or = to indicate the pattern of the simple effect of Task Presentation for Hard Tasks > 3) Is there an interaction? Is the simple effect of Task Presentation the same for Easy and for Hard Tasks ??? 4) Describe the pattern of the interaction...

3. Review #1-- 1st main effects Task Presentation Paper Computer Task Difficulty Easy 90 70 Hard 40 60 1) Compute the marginal means for Task Presentation. 2) Put in a <, > or = to indicate the pattern of the main effect of Task Presentation > > 3) Is there a main effect of Task Presentation ??? 65 65 = 4) Is the pattern of the main effect for Task Presentation descriptive or potentially misleading (is the pattern of the main effect or Task Presentation the same as the patterns of the simple main effects of Task Presentations for both Easy and Hard Tasks) ? 5) Describe the main effect

4. Review #1-- 2nd main effects Task Presentation Paper Computer Task Difficulty Easy 90 70 Hard 40 60 1) Compute the marginal means for Task Difficulty 2) Put in a <, > or = to indicate the pattern of the main effect of Task Difficulty 80 50 > > > 3) Is there a main effect of Task Difficulty ??? 4) Is the pattern of the main effect of Task Difficulty descriptive or potentially misleading (is the pattern of the main effect or Task Difficulty the same as the patterns of the simple main effects of Task Difficulty for both Computer and Paper Presentations) ? 5) Describe the main effect

5. “Kinds” of 2-factor Designs • BETWEEN GROUPS FACTORIAL DESIGN: • each IV uses a between groups comparison • each participant completes only one condition of the design • WITHIN-GROUPS FACTORIAL DESIGN: • each IV uses a within-groups comparison • each participant completes all conditions of the design • MIXED FACTORIAL DESIGN: • one IVs uses a between groups comparison and the other IV uses a within-groups comparison. • each participant completes both conditions of the within- groups IV, but completes only one condition of the between groups IV. • it is important to specify which IV is “BG” and which is “WG”

6. Between groups factorial design  experimental or natural grps designs used to study “differences” Each participant is in only one condition, having a particular combination of Initial Diagnosis and Type of Treatment. Type of Treatment Initial Diagnosis Individual Group Therapy Therapy Clients diagnosed Clients diagnosed Depression as depressed who as depressed who are treated with are treated with individual therapy group therapy Clients diagnosed Clients diagnosed Social Anxiety with social anxiety with social anxiety who are treated with who are treated with individual therapy group therapy

7. Mixed group factorial design  natural groups designs used to study “different changes” or “changing differences” Species was a between groups IV (a turtle can only be a member of one species). Each turtle participated in both the mid-morning & dusk conditions of the Time of Day IV. Species of Turtle Time of Day Snapping Turtle Painted Turtle Each snapping turtle Each painted turtle Mid-morning completed a trial completed a trial during mid-morning during mid-morning Each snapping turtle Each painted turtle Evening completed a trial completed a trial during the evening during the evening

8. Mixed group factorial design  experimental designs used to increase data collection efficiency or statistical power Type of Evidence was a between groups IV -- people can’t read the “same study” twice & give independent ratings. Each participant rated the guilt of both Defendants -- a within-groups IV --as they would in this type of case. Type of Evidence Defendant DNA Eye Witness Major Actor DNA evidence was An eye witness testified presented against to seeing the major the major actor commit the crime Conspirator DNA evidence was An eye witness testified presented against to seeing the conspirator the conspirator commit the crime

9. Within-groups factorial design – experimental designs used to increase data collection efficiency or statistical power Each participant completed four trials, one of each combination of Retention Interval and Word Type. Retention Interval Word Type Immediate Test Delayed Test The test was given The test was given Familiar immediately after the 5 minutes after the study of a list of study of a list of 40 familiar words. 40 familiar words. The test was given The test was given Unfamiliar immediately after the 5 minutes after the study of a list of study of a list of 40 unfamiliar words. 40 unfamiliar words.

10. Within-groups factorial design – natural designs to study “changing changes” Each participant was observed in both School & Home (WG) settings both when they were 12 &16 (WG) Age Setting 12 years old 16 years old School Participants were Participants were observed in a school observed in a school setting at age 12. setting at age 16. Home Participants were Participants were observed in a home observed in a home setting at age 12. setting at age 16.

11. Practice Identifying Types of Factorial Designs - answers next page The purpose of the study was to examine the possible influence of two variables upon maze-learning by rats, length of the maze (either 10 feet or 30 feet) and the size of the reward (either 1sugar pellet or 5 sugar pellets). Here are three “versions” of the study tell which is BG, WG & MG a. Each rat completed one trial. Each was assigned to either the longer or the shorter maze, and also assigned to receive either 1 or 5 sugar pellets upon completing the maze. b. Each rat completed two trials in either the longer or the shorter maze. Following one trial in the assigned maze, each received 1 pellet reward, after the other trial they received the 5 pellets. c. Each rat completed four trials, two in the shorter maze and two in the longer maze. Each received 1 pellet after one of the short-maze trials and 5 pellets after the other, and also 1 pellet after one of the long-maze trials and 5 pellets after the other. BG MG WG

12. Another Example -- 3 versions of the same study The researcher wanted to investigate infant's startle responses to loud sounds. The two variables of interest were the Position of the Sound (in front of versus behind the infant) and the Type of Sound (a hand-clap versus deep male voice saying "Hey"). Here are three “versions” of the study tell which is BG, WG & MG a. Each infant completed trials all involving a hand-clap or all involving the voice saying "Hey". During some of the trials, the appropriate type of sound was made in front of the infant. During other trials, the appropriate type of sound was made behind the infant. b. Each infant had some trials during which the sound was made in front of then and some during which the sound was made behind them. Some of the sounds were the hand-clap and the others were the voice saying "Hey". c. Each infant always heard either the hand-clap or the “Hey”, and whatever sound they heard was always played either in front of them or behind them. MG WG BG

13. Remember about the causal interpretation of effects of a factorial design Start by assessing the causal interpretability of each main effect Remember, in order to causally interpret an interaction, you must be able to casually interpret BOTH main effects. For each of the following: Tell the IVs and tell what effects could be causally interpreted (assuming proper RA, IV manip. and confound control were used): 1. Male and female participants who were African American, Mexican American, or European American were asked to complete a questionnaire about satisfaction with their Senators. 2. Children played with either a toy gun, a toy car or a puzzle, some while their parents were in the room and some not. The DV was the amount of aggressive behavior they exhibited. 3. Participants played with either a simple puzzle or a complex puzzle in pairs made up of two boys, two girls or one boy & one girl. zilcho-causo All three ! Puzzle type only.

14. Statistical Analysis of 2x2 Factorial Designs • Like a description of the results based upon inspection of the means, formal statistical analyses of factorial designs has five basic steps: • 1. Tell IVs and DV 2. Present data in table or figure • 3.Determine if the interaction is significant • if it is, describe it in terms of one of the sets of simple effects. • 4.Determine whether or not the first main effect is significant • if it is, describe it • determine if that main effect is descriptive or misleading • 5. Determine whether or not the second main effect is significant • if it is, describe it • determine if that main effect is descriptive or misleading

15. Interpreting Factorial Effects • Important things to remember: • main effects and the interaction are 3 separate effects each must be separately interpreted -- three parts to the “story” • most common error -- “interaction is different main effects” • best thing -- be sure to carefully separate the three parts of the story and tell each completely • Be careful of “causal words” when interpreting main effects and interactions (only use when really appropriate). • caused, effected influenced, produced, changed …. • Consider more than the “significance” • consider effect sizes, confidence intervals, etc. when describing the results

16. Statistical Analysis of a 2x2 Design • Task Presentation (a)SE of Presentation • Paper Computer for Easy Tasks • Task Difficulty (b) Easy 90 7080 • Hard40 6050 • 65 65 SE for Presentation for Hard Tasks • PresentationDifficultyInteractionMain EffectMain EffectEffect • SSPresentationSSDificulty SSInteraction • 65 vs. 6580 vs. 50SEEasy vs. SEHard

17. Constructing F-tests for a 2x2 Factorial • FPresentation =( SSPresentation / dfPresentation ) • ( SSError / dfError) • FDifficulty=( SSDifficulty/ dfDifficulty ) • ( SSError / dfError ) • FInteraction=( SSInteraction/ dfInteraction ) • ( SSError / dfError)

18. Statistical Analyses Necessary to Describe Main Effects of a 2x2 Design • In a 2x2 Design, the Main effects F-tests are sufficient to tell us about the relationship of each IV to the DV… • since each main effect involves the comparison of two marginal means -- the corresponding significance test tells us what we need to know … • whether or not those two marginal means are “significantly different” • Don’t forget to examine the means to see if a significant difference is in the hypothesized direction !!!

19. Statistical Analyses Necessary to Describe the Interaction of a 2x2 Design • However, the F-test of the interaction only tells us whether or not there is a “statistically significant” interaction… • it does not tell use the pattern of that interaction • to determine the pattern of the interaction we have to compare the simple effects • to describe each simple effect, we must be able to compare the cell means • we need to know how much of a cell mean difference is “statistically significant”

20. Using LSD to Compare cell means to describe the simple effects of a 2x2 Factorial design • LSD can be used to determine how large of a cell mean difference is required to treat it as a “statistically significant mean difference” • Will need to know three values to use the computator • dferror -- look on the printout or use N – 4 • MSerror – look on the printout • n = N / 4 -- use the decimal value – do not round to the nearest whole number! Remember – only use the lsdmmd to compare cell means. Marginal means are compared using the man effect F-tests.

21. Applying lsdmmd to 2x2 BG ANOVA • Task Presentation • Paper Computer • Task Difficulty for the interaction Easy 60 90F(1,56) = 6.5, p = .023 • Hard60 70 lsdmmd = 14 • Is there a Task Difficulty main effect? Based on what? • for the following, tell the mean difference and apply the lsdmmd Yes! F-test of Int 30 > Simple effect of Task Presentation SE of Task Presentation for Easy Tasks SE of Task Presentation for Hard Tasks Simple effects of Task Difficulty SE of Task Difficulty for Paper Pres. SE of Task Difficulty for Comp. Pres. 10 = 0 = 20 >

22. Applying lsdmmd to 2x2 BG ANOVA • Task Presentation • Paper Computer • Task Difficulty for Difficulty ME • Easy 60 90 75F(1,56) = 4.5, p = .041 • Hard60 70 65 lsdmmd = 14 • Is there a Task Difficulty main effect? Based on what? • Is main effect descriptive (unconditional) or potentially misleading (conditional)? Yes! F-test of ME Simple effects of Task Difficulty SE of Task Difficulty for Paper Pres. SE of Task Difficulty for Comp. Pres. 0 = 20 > Descriptive only for Computer presentation; misleading for Paper presentations.

23. Applying lsdmmd to 2x2 BG ANOVA • Task Presentation • Paper Computer • Task Difficulty for Presentation ME • Easy 60 90F(1,56) = 7.2, p = .011 • Hard60 70 lsdmmd = 14 • 60 80 • Is there a Task Difficulty main effect? Based on what? • Is main effect descriptive (unconditional) or potentially misleading (conditional)? Yes! F-test of ME Simple effects of Task Difficulty SE of Task Presentation for Easy Tasks SE of Task Presentation for Hard Tasks 30 < 10 = Descriptive only for Easy tasks; misleading for Difficult tasks.

24. Effect Sizes for 2x2 BG Factorial designs • For Main Effects & Interaction (each w/ df=1) • r =  [ F / (F + dferror)] • For Main Effects & Simple Effects • d = (M1 - M2 ) /  Mserror • d² • r = ---------- •  d² + 4 (This is an “approximation formula”)