Experimental Control & Design

1. Experimental Control& Design Psych 231: Research Methods in Psychology

2. Don’t forget, in labs • IRB worksheet for group projects • Methods and Appendix for group projects Announcements

3. Methods of Experimental Control • Constancy/Randomization • Comparison • Production Controlling Variability

4. Constancy/Randomization • If there is a variable that may be related to the DV that you can’t (or don’t want to) manipulate • Control variable: hold it constant • Random variable: let it vary randomly across all of the experimental conditions • But beware of potential confounds, variables that co-vary with both the IV and DV but aren’t controlled Methods of Controlling Variability

5. Comparison • An experiment always makes a comparison, so it must have at least two groups • Sometimes there are control groups • This is typically the absence of the treatment Training group No training (Control) group • Without control groups if is harder to see what is really happening in the experiment • It is easier to be swayed by plausibility or inappropriate comparisons Methods of Controlling Variability

6. Comparison • An experiment always makes a comparison, so it must have at least two groups • Sometimes there are control groups • This is typically the absence of the treatment • Sometimes there are a range of values of the IV 1 week of Training group 2 weeks of Training group 3 weeks of Training group Methods of Controlling Variability

7. Production • The experimenter selects the specific values of the Independent Variables 1 week of Training group 2 weeks of Training group 3 weeks of Training group • Need to do this carefully • Suppose that you don’t find a difference in the DV across your different groups • Is this because the IV and DV aren’t related? • Or is it because your levels of IV weren’t different enough Methods of Controlling Variability

8. So far we’ve covered a lot of the about details experiments generally • Now let’s consider some specific experimental designs. • Some bad designs • Some good designs • 1 Factor, two levels • 1 Factor, multi-levels • Between & within factors • Factorial (more than 1 factor) Experimental designs

9. Bad design example 1: Does standing close to somebody cause them to move? • “hmm… that’s an empirical question. Let’s see what happens if …” • So you stand closely to people and see how long before they move • Problem: no control group to establish the comparison group (this design is sometimes called “one-shot case study design”) Poorly designed experiments

10. Bad design example 2: • Testing the effectiveness of a stop smoking relaxation program • The participants choose which group (relaxation or no program) to be in Poorly designed experiments

11. Random Assignment • Bad design example 2: Non-equivalent control groups Self Assignment Independent Variable Dependent Variable Training group Measure participants No training (Control) group Measure Problem: selection bias for the two groups, need to do random assignment to groups Poorly designed experiments

12. Bad design example 3:Does a relaxation program decrease the urge to smoke? • Pretest desire level – give relaxation program – posttest desire to smoke Poorly designed experiments

13. Pre-test No Training group Post-test Measure • Bad design example 3: One group pretest-posttest design Dependent Variable Independent Variable Dependent Variable participants Pre-test Training group Post-test Measure Add another factor • Problems include: history, maturation, testing, and more Poorly designed experiments

14. Good design example • How does anxiety level affect test performance? • Two groups take the same test • Grp1 (moderate anxiety group): 5 min lecture on the importance of good grades for success • Grp2 (low anxiety group): 5 min lecture on how good grades don’t matter, just trying is good enough • 1 Factor (Independent variable), two levels • Basically you want to compare two treatments (conditions) • The statistics are pretty easy, a t-test 1 factor - 2 levels

15. Random Assignment Dependent Variable Anxiety Low Test participants Moderate Test • Good design example • How does anxiety level affect test performance? 1 factor - 2 levels

16. One factor Use a t-test to see if these points are statistically different test performance low moderate low moderate anxiety Two levels • Good design example • How does anxiety level affect test performance? anxiety 60 80 Observed difference between conditions T-test = Difference expected by chance 1 factor - 2 levels

17. Advantages: • Simple, relatively easy to interpret the results • Is the independent variable worth studying? • If no effect, then usually don’t bother with a more complex design • Sometimes two levels is all you need • One theory predicts one pattern and another predicts a different pattern 1 factor - 2 levels

18. Interpolation What happens within of the ranges that you test? test performance low moderate anxiety • Disadvantages: • “True” shape of the function is hard to see • Interpolation and Extrapolation are not a good idea 1 factor - 2 levels

19. Extrapolation What happens outside of the ranges that you test? test performance low moderate anxiety high • Disadvantages: • “True” shape of the function is hard to see • Interpolation and Extrapolation are not a good idea 1 factor - 2 levels

20. For more complex theories you will typically need more complex designs (more than two levels of one IV) • 1 factor - more than two levels • Basically you want to compare more than two conditions • The statistics are a little more difficult, an ANOVA (Analysis of Variance) 1 Factor - multilevel experiments

21. Grp3 (high anxiety group): 5 min lecture on how the students must pass this test to pass the course • Good design example (similar to earlier ex.) • How does anxiety level affect test performance? • Two groups take the same test • Grp1 (moderate anxiety group): 5 min lecture on the importance of good grades for success • Grp2 (low anxiety group): 5 min lecture on how good grades don’t matter, just trying is good enough 1 Factor - multilevel experiments

22. Random Assignment Dependent Variable Anxiety Low Test participants Moderate Test High Test 1 factor - 3 levels

23. anxiety mod high low test performance 60 80 low mod high anxiety 60 1 Factor - multilevel experiments

24. Advantages • Gives a better picture of the relationship (function) • Generally, the more levels you have, the less you have to worry about your range of the independent variable 1 Factor - multilevel experiments

25. 2 levels 3 levels testperformance test performance low mod high low moderate anxiety anxiety Relationship between Anxiety and Performance

26. Disadvantages • Needs more resources (participants and/or stimuli) • Requires more complex statistical analysis (analysis of variance and pair-wise comparisons) 1 Factor - multilevel experiments

27. The ANOVA just tells you that not all of the groups are equal. • If this is your conclusion (you get a “significant ANOVA”) then you should do further tests to see where the differences are • High vs. Low • High vs. Moderate • Low vs. Moderate Pair-wise comparisons