research design

1. Research Design October 22, 2007

3. How to select a research design: What is study purpose? To describe vs. to predict/evaluate How many data collection points are needed? Cross-sectional vs. longitudinal How many groups are needed? One group Two groups who receive different tx More than two groups

4. Validity in Research Design The extent to which a study allows causal inferences to be made about relationships between variables. To infer causality requires: statistical association between independent and dependent variables; temporality; relationship must not be spurious.

5. Two types of validity in research design External validity- generalizability; Will we find same results with different people in a different place at a different time? Internal validity How sure are we that the results were caused by the independent variable and not something else?

6. Threats to internal validity History- other events in time Maturation Testing Instrumentation changes Regression to the mean Selection biases/ between group differences Ambiguity in direction of causal inferences History: 9/11 study of firefighter depression; Maturation: kids grow up and become less impacted by peers; Testing- ??? Ambiguity: did group prevent hs dropout or did staying in school predict group attendance?History: 9/11 study of firefighter depression; Maturation: kids grow up and become less impacted by peers; Testing- ??? Ambiguity: did group prevent hs dropout or did staying in school predict group attendance?

7. Campbell & Stanley Experiment Notation X O R

8. The simplest research design: 01 Examples: Qualitative study Descriptive study Exploratory study NOT the right design if purpose is to: Evaluate intervention�s effectiveness Compare one group to another

9. Pre-experimental designs X O One-shot case study Example studies: What was delinquency rates of kids who attended afterschool program? What is organ donation rate among people who received an info brochure? No correlation between variables. Not able to rule out spurious causes. O1 X O2 One-group pretest-posttest design Example studies: Did attending afterschool program decrease youth truancy rate? Did attitudes toward organ donation improve after reading brochure? Not able to rule out spurious causes.

10. Quasi-experimental design X O O Post-test only with non-equivalent groups Example: Did DQ rates differ between youth attending after school program and non-attenders? Do people who read organ donation brochure report higher donation willingness than those that did not? Not able to rule out spurious causes. Two groups may be different in unmeasured ways.

11. True experimental design R X O R O Post-test only control group design Because of random assignment, this is experimental design.

12. Other Experimental Designs R O1 X O2 R O1 O2 Pretest posttest control group design Does not control for testing/ retesting effects. R O1 X O2 R O1 O2 R X O2 R O2 Solomon four-group design

13. Testing alternative treatments R O1 XA O2 R O1 XB O2 R O1 O2 Example: Which afterschool program is most effective at decreasing youth delinquency: Boys & Girls Club, the YMCA, or not attending any afterschool program? Compared to no intervention, is distributing a brochure or showing a video more effective at promoting organ donation decisions? Alternative treatment design with pretest R O1 XAB O2 R O1 XA O2 R O1 XB O2 R O1 O2 Example: Did people who received a brochure and watch a video show greater intention to donate organs than people who saw only the video, received only the brochure, or had no intervention? Dismantling studies

14. Randomization Random selection vs. random assignment Sampling Research design

15. Quasi-experimental designs Use comparison group (not randomly selected) rather than control group (randomly selected) These are COMMON in social science research because some things are hard to randomize

16. Techniques to improve validity of quasi-experimental designs Use multiple pre-test measures O1 O2 X O3 O1 O2 O3 Use switching replications O1 X O2 O3 O1 O2 X O3 Select a comparison group as similar as possible to experimental group

17. Time-series Designs Simple O1 O2 O3 O4 X O5 O6 O7 Example: How did introduction of sentencing juveniles as adults impact offending rates in MD? Multiple O1 O2 O3 O4 X O5 O6 O7 O1 O2 O3 O4 O5 O6 O7 Example: Compared to a state that has not initiated a policy about juvenile sentencing, how did introduction of sentencing juveniles as adults impact offending rates in MD?

18. Thinking about your project� Person on Left: share your research question and proposed research design. Person on Right: Say, �That sounds great, but have you thought about [alternative research design]?� Switch roles.

19. Top 9 Pitfalls that design logic can�t fix Experimenter Expectancies Obtrusive observation Novelty effects Placebo effects Control group contamination Diffusion of Treatment Attrition Resistance to case assignment protocol Measurement bias

20. Small Group Activity: Using Empirical Article Samples What research design was used in study? Write Campbell & Stanley notation for it. If the study was not a true experimental design, discuss whether it would be possible to design an experiment to answer the research question.

21. Next Week: Qualitative Research I Required Reading: MS&D Ch 9 SELECT ONE EXAMPLE FROM BELOW TO READ: Burstein, Bryan & Chao, 2005: Youth with special needs; Mohatt et al., 2004: Alcoholism among Alaska natives; Morgan et al., 2004: Youth hustling

22. Guest Speaker