Research Design Part II: Cross-sectional and Quasi-Experimental Designs

1. Research Design Part II: Cross-sectional and Quasi-Experimental Designs • Frankfort-Nachmias & Nachmias (Chapter 6 – Cross-Sectional and Quasi-Experimental Designs) • Gerring (Chapter 8) • Campbell and Stanley, “Experimental and Quasi-Experimental Designs for Research.” (remainder) • King, Keohane and Verba (Chapter 5, sections 5.2 - 5.6) • Applications • Stack, S. and Gunlach, J. (1992) “The Effect of Country Music on Suicide.”Social Forces 71: 211–18. • Lawrence S. Rothenberg; Mitchell S. Sanders, “Severing the Electoral Connection: Shirking in the Contemporary Congress.” American Journal of Political Science, Vol. 44, No. 2. (Apr., 2000), pp. 316-325. (Difference-in-difference)  

2. Writing a Literature Review • Typical format for research article • Introduction • Literature Review • Theory • Research Design • Empirical Analysis and Results • Conclusion

3. Writing a Literature Review • Purpose of literature review • Inform reader of prior relevant work • Persuade reader that your work is important (justify your research)

4. Possible Justifications for Your Research • New question, new theory • New question, existing theory • Old question, new theory • Old question, conflicting theories • Old question, conflicting findings • Old question, new methods • Old question, new data

5. Literature Review Don’ts • Don’t (just) provide a chronological listing of article summaries • Don’t provide every detail of every study

6. Literature Review Do’s • Organize your discussion of the literature in a way that reflects and supports the justification for your research • Provide more detail for seminal studies, less detail (or simply a citation) for less cited studies • For questions that have been studied extensively, it is not necessary to cite every study • End your literature review with a summary and critique that justifies your research

7. Literature Review Assignment • Approx. 7-10 pages, double-spaced • Due November 2nd

8. How to Identify the Relevant Literature • Use electronic databases and keyword searches (Google Scholar, JSTOR) • Prioritize: • Articles • Articles published in highly-ranked journals • Recently published articles • Seminal articles • Number of articles: 10-20?

9. Quasi-experimental and Cross-Sectional Designs • What are they? • Quasi-experimental – study of more than one sample (often over period of time) • Cross-sectional – Analysis of a single sample (lacks random assignment, temporal variation, and manipulation); but includes comparison groups • Pre-experimental – Cross-sectional, with no comparison group; causal inference impossible

10. Quasi-experimental and Cross-Sectional Designs • Why? • Property-disposition relationship • vs. • Stimulus-response relationship

11. Pre-Experimental Designs • One-Shot Case Study • One group • No variation in independent variable X O

12. Pre-Experimental Designs • Example: • Dependent variable: Americans’ support for campaign finance reform • Independent variable: Watergate scandal • Data: 1976 survey of American adults; examine mean level of support

13. Cross-Sectional Designs • Static-Group Comparison Design • Two groups – observed at one time • Allows variation in the independent variable X O1 O2

14. Elaboration of Static Group Comparison Design • Correlational / Cross-Sectional Designs X1 O1 X2 O2 X3 O3 X4 O4 Xi Oi • Problems with correlational/cross-sectional designs?

15. Example: Wine and Health • Hypothesis: Drinking wine causes individuals to be healthier (esp. heart) • Existing studies: compared the health of wine drinkers to the health of those who do not drink wine: Research design X1 (Wine drinkers) O1 (Health) X2 (Non-drinkers) O2 (Health)

16. Spurious Results?

17. Controlling for Affluence Research design: X1 (Affluent Wine drinkers) O1 (Health) X2 (Affluent Non-drinkers) O2 (Health) X3 (Poor Wine drinkers) O3 (Health) X4 (Poor Non-drinkers) O4 (Health)

18. Another Example (?) • http://www.washingtonpost.com/wp-dyn/content/article/2006/05/25/AR2006052501729.html • http://www.sciencedaily.com/releases/2007/04/070417193338.htm

19. Quasi-Experimental Designs • Contrasted Groups Design • Multiple groups, based on some categorical variable • Observed at one point in time (similar to cross-sectional) O1 O2 O3 O4 Oi • Problems with contrasted groups designs?

20. Quasi-Experimental Designs • One-Group Pretest-Posttest Design • One group • Allows variation in the independent variable O1 X O2

21. Elaborations of the One-Group Pretest-Posttest Design • Time Series Designs • simple vs. extended O1…OkX Ok+1…Om k = # of pretest observations m = total # observations Or (“equivalent time samples design”) X1 O1X2 O2X3O3 … Xm Om

23. Example: Murray’s “Poverty-Spending Paradox” (Schram 1991)

24. Example: Murray’s “Poverty-Spending Paradox” (Schram 1991)

25. Nonequivalent Control Group Design • No random assignment O1 X O2 O3 O4 or O1 X1 O2 O3 X2 O4

26. Control Series Designs • Addition of second (control) group to time series design (CS: “multiple time series design”) O1 O2 O3 X1 O4 O5 O6…… O7 O8 O9 X2 O10 O11 O12……

27. Panel Designs • Repeated observations of the same units over time • Also goes by: • Pooled time-series design • Pooled cross-sectional time-series design

28. The Effect of Sanctioning in the TANF Program

29. The Effect of Sanctioning in the TANF Program

32. Donahue and Levitt • Hypothesis: The (legal) availability of abortion in a state is negatively related to the crime rate (many years later). O1 O2 O3 X1(early legalization) O4 O5 O6… O7 O8 O9 X2 (legalize 1973) O10 O11 O12…

34. Gerring’s Criteria for Research Design • Plentitude (N) • Boundedness (relevant cases) • Comparability (descriptive/causal) • Independence • Representativeness • Variation (X, Y, X&Y) • Analytic Utility (of the sample) • Replicability • Mechanism • Causal Comparison

35. KKV – Overcoming Common Problems • Omitted Variable Bias • Inclusion of Irrelevant Variables • Endogeneity • Assigning Values of the IV’s • Controlling the Research Situation

36. Does Country Music Cause Suicide?

37. Stack & Gundlach • Hypothesis: There is a positive relationship between exposure to country music and suicide rates • Research design: X1 (no country music) O1 (suicide rate) X2 (1 station) O2 (suicide rate) X3 (2 stations) O3 (suicide rate) X4 (3 stations) O4 (suicide rate) Xi ( etc.) Oi (suicide rate)

38. Stack & Gundlach • Findings: 51% of the variation in urban white suicide rates can be explained by variation in airtime devoted to country music • Internal Validity?

45. O1ARetiredAO4A O2B SW-OfficeB O5B O3CReturnedC O6C

47. Regression Discontinuity Designs • Example: What is the effect of an award on later achievement? (from CS)

48. Regression Discontinuity Designs • Example: What is the effect of an award on later achievement? (from CS) • Inferential challenge: Award recipients are likely to do well anyway, even without the award, because criteria for receiving award also predict future success

49. A regression discontinuity design is appropriate for any research design in which the assignment of the treatment is determined by a continuous variable that is also related to the outcome of interest.

50. Ludwig and Miller, 2007