1 / 65

The Plan for Day Two

The Plan for Day Two. Practice and pitfalls (1) Natural experiments as interesting sources of instrumental variables (2) The consequences of “weak” instruments for causal inference (3) Some useful IV diagnostics (4) Walk through an empirical application

Albert_Lan
Télécharger la présentation

The Plan for Day Two

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Plan for Day Two • Practice and pitfalls (1) Natural experiments as interesting sources of instrumental variables (2) The consequences of “weak” instruments for causal inference (3) Some useful IV diagnostics (4) Walk through an empirical application • Goal = provide concrete examples of instrumental variables methods

  2. Instrumental Variables and Natural Experiments • What is a natural experiment? • “situations where the forces of nature or government policy have conspired to produce an environment somewhat akin to a randomized experiment” • Angrist and Krueger (2001, p. 73) • Natural experiments can provide a useful source of exogenous variation in problematic regressors • But they require detailed institutional knowledge

  3. Instrumental Variables and Natural Experiments • Some natural experiments in economics • Existing policy differences, or changes that affect some jurisdictions (or groups) but not others • Minimum wage rate • Excise taxes on consumer goods • Unemployment insurance, workers’ compensation • Unexpected “shocks” to the local economy • Coal prices and the Middle East oil embargo (1973) • Agricultural production and adverse weather events

  4. Instrumental Variables and Natural Experiments • Some potential pitfalls • Not all policy differences/changes are exogenous • Political factors and past realizations of the response variable can affect existing policies or policy changes • Generalizability of causal effect estimates • Results may not generalize beyond the units under study • Heterogeneity in causal effects • Results may be sensitive to the natural experiment chosen in a specific study (L.A.T.E.)

  5. Instrumental Variables and Natural Experiments • Some natural experiments of criminological interest • Levitt (1996) = prison population → crime rate • Levitt (1997) = police hiring → crime rate • Apel et al. (2008) = youth employment → delinquency • Some natural experiments not of criminological interest, but interesting nonetheless • Angrist and Evans (1998) = fertility → labor supply

  6. Levitt (1996), Q.J.E. • Large decline in crime did not accompany the large increase in prison population (1971-1993) • Prima fascia evidence of prison ineffectiveness • But...increased prison use could mask what would have been a greater increase in crime • Underlying determinants of crime probably worsened • And...prison population probably responded to crime increase

  7. Levitt (1996), Q.J.E. • Prison overcrowding legislation • Population caps, prohibition of “double celling” • In 12 states, the entire prison system came under court control • AL, AK, AR, DE, FL, MS, NM, OK, RI, SC, TN, TX • Relationship between legislation and prisons • Prior to filing, prison growth outpaced national average by 2.3 percent • After filing, prison growth was 5.1 percent slower

  8. – – Prisons Under Court Control Prison Population Growth Crime Rate Growth Levitt (1996), Q.J.E. • Logic of the instrumental variable in this study • Court rulings concerning prison capacity cannot be correlated with the unobserved determinants of crime rate changes • Or...the only reason court rulings are related to crime is because they limit prison population growth

  9. Levitt (1996), Q.J.E. • 2SLS model yields a “prison effect” on crime at least four times as high as the LS model • Violent crime rate • bLS = –.099 (s.e. = .033) • bIV = –.424 (s.e. = .201) • Property crime rate • bLS = –.071 (s.e. = .019) • bIV = –.321 (s.e. = .138) • A 10% increase in prison size produces a 4.2% decrease in violent crime and a 3.2% decrease in property crime

  10. Levitt (1996), Q.J.E. • L.A.T.E. = effect of prison growth on crime among states under court order to slow growth • Some relevant observations • Generalizability = predominately Southern states • Large prison populations, unusually fast prison growth • T.E. heterogeneity = (slowed) prison growth due to court-ordered prison reductions may be differentially related to crime rates • Other IV’s could lead to different causal effect estimates

  11. Levitt (1997), A.E.R. • Breaking the simultaneity in the police-crime connection • When more police are hired, crime should decline • But...more police may be hired during crime waves • Election cycles and police hiring • Increases in size of police force disproportionately concentrated in election years • Growth is 2.1% in mayoral election years, 2.0% in gubernatorial election years, and 0.0% in non-election years

  12. + – Growth in Police Manpower Growth in Crime Rate Election Year Levitt (1997), A.E.R. • However...can election cycles affect crime rates through other spending channels? • Ex., education, welfare, unemployment benefits • If so, all of these other indirect channels must be netted out

  13. Levitt (1997), A.E.R. Reduced-form coefficients First-stage coefficients

  14. Levitt (1997), A.E.R. • Comparative estimates of the effect of police manpower on city crime rates • Violent crime rate • Levels: bLS = +.28 (s.e. = .05) • Changes: bLS = –.27 (s.e. = .06) • Changes: bIV = –1.39 (s.e. = .55) • Property crime rate • Levels: bLS = +.21 (s.e. = .05) • Changes: bLS = –.23 (s.e. = .09) • Changes: bIV = –.38 (s.e. = .83)

  15. Levitt (1997), A.E.R. • Follow-up instrumental variables studies of the police-crime relationship in the U.S. • Levitt (2002) = Number of firefighters • Klick and Tabarrok (2005) = Washington, DC, terrorism alert levels post-9/11 • Evans and Owens (2007) = Grants from the federal Office of C.O.P.S. • These findings basically replicated those from Levitt’s (1997) original study

  16. Apel et al. (2008), J.Q.C. • What effect does working have on adolescent behavior? • Prior research suggests the consequences of work are uniformly negative • Focus on “work intensity” rather than work per se • Youth Worker Protection Act • Problem of non-random selection into youth labor market • Especially pronounced for high-intensity workers

  17. Apel et al. (2008), J.Q.C. • Something interesting happens at age 16 • Youth work is no longer governed by the federal Fair Labor Standards Act (F.L.S.A.)

  18. Apel et al. (2008), J.Q.C. • F.L.S.A. governs employment of all 15 year olds during the school year • No work past 7:00 pm • Maximum 3 hours/day and 18 hours/week • But, F.L.S.A. expires for 16 year olds • And...every state has its own law governing 16-year-old employment • Thus, youth age into less restrictive regimes that vary across jurisdictions

  19. Apel et al. (2008), J.Q.C. • Change in work intensity at 15-16 transition among 15-year-old non-workers Magnitude of change is an increasing function of the number of hours allowed at age 16

  20. Apel et al. (2008), J.Q.C.

  21. Apel et al. (2008), J.Q.C. • A 20-hour increase in the number of hours worked per week reduces the “variety” of delinquent behavior by 0.47 (–.023320)

  22. Angrist and Krueger (1991), J.L.E. • Returns to education (Y = wages) • Problem of omitted “ability bias” • Years of schooling vary by quarter of birth • Compulsory schooling laws, age-at-entry rules • Someone born in Q1 is a little older and will be able to drop out sooner than someone born in Q4 • Q.O.B. can be treated as a useful source of exogeneity in schooling

  23. Source: Angrist and Krueger (1991), Figure I Angrist and Krueger (1991), J.L.E. • People born in Q1 do obtain less schooling • But pay close attention to the scale of the y-axis • Mean difference between Q1 and Q4 is only 0.124, or 1.5 months • So...need large N since R2X,Z will be very small • A&K had over 300k for the 1930-39 cohort

  24. Angrist and Krueger (1991), J.L.E. • Final 2SLS model interacted QOB with year of birth (30), state of birth (150) • OLS: b = .0628 (s.e. = .0003) • 2SLS: b = .0811 (s.e. = .0109) • Least squares estimate does not appear to be badly biased by omitted variables • But...replication effort identified some pitfalls in this analysis that are instructive

  25. Bound, Jaeger, and Baker (1995), J.A.S.A. • Potential problems with QOB as an IV • Correlation between QOB and schooling is weak • Small Cov(X,Z) introduces finite-sample bias, which will be exacerbated with the inclusion of many IV’s • QOB may not be completely exogenous • Even small Cov(Z,e) will cause inconsistency, and this will be exacerbated when Cov(X,Z) is small • QOB qualifies as a weak instrument that may be correlated with unobserved determinants of wages (e.g., family income)

  26. Bound, Jaeger, and Baker (1995), J.A.S.A. • Even if the instrument is “good,” matters can be made far worse with IV as opposed to LS • Weak correlation between IV and endogenous regressor can pose severe finite-sample bias • And…really large samples won’t help, especially if there is even weak endogeneity between IV and error • First-stage diagnostics provide a sense of how good an IV is in a given setting • F-test and partial-R2 on IV’s

  27. Useful Diagnostic Tools for IV Models • Tests of instrument relevance • Weak IV’s → Large variance of bIV as well as potentially severe finite-sample bias • Tests of instrument exogeneity • Endogenous IV’s → Inconsistency of bIV that makes it no better (and probably worse) than bLS • Durbin-Wu-Hausman test • Endogeneity of the problem regressor(s)

  28. Tests of Instrument Relevance • Diagnostics based on the F-test for the joint significance of the IV’s • Nelson and Startz (1990); Staiger and Stock (1997) • Bound, Jaeger, and Baker (1995) • Partial R-square for the IV’s • Shea (1997) • There is a growing econometric literature on the “weak instrument” problem

  29. Tests of Instrument Exogeneity • Model must be overidentified, i.e., more IV’s than endogenous X’s • H0: All IV’s uncorrelated with structural error • Overidentification test: 1. Estimate structural model 2. Regress IV residuals on all exogenous variables 3. Compute NR2 and compare to chi-square • df = # IV’s – # endogenous X’s

  30. Durbin-Wu-Hausman (DWH) Test • Balances the consistency of IV against the efficiency of LS • H0: IV and LS both consistent, but LS is efficient • H1: Only IV is consistent • DWH test for a single endogenous regressor: DWH = (bIV – bLS) / √(s2bIV – s2bLS) ~ N(0,1) • If |DWH| > 1.96, then X is endogenous and IV is the preferred estimator despite its inefficiency

  31. Durbin-Wu-Hausman (DWH) Test • A roughly equivalent procedure for DWH: 1. Estimate the first-stage model 2. Include the first-stage residual in the structural model along with the endogenous X 3. Test for significance of the coefficient on residual • Note: Coefficient on endogenous X in this model is bIV (standard error is smaller, though) • First-stage residual is a “generated regressor”

  32. Software Considerations • I have a strong preference for Stata • Classic routine (-ivreg-) as well as a user-written one with a lot more diagnostic capability (-ivreg2-) • Non-linear models: -ivprobit- and -ivtobit- • Panel models: -xtivreg- and -xtivreg2- • Useful post-estimation routines • Overidentification: -overid- • Endogeneity of X in LS model: -ivendog- • Heteroscedasticity: -ivhettest-

  33. Software Considerations • Basic model specification in Stata ivreg y (x = z) w [weight = wtvar], options y = dependent variable x = endogenous variable z = instrumental variable w = control variable(s) • Useful options: first, ffirst, robust, cluster(varname)

  34. Software Considerations • For SAS users: Proc Syslin (SAS/ETS) • Basic command: proc syslin data=dataset2sls options1; endogenous x; instruments z w; model y = x w/options2; weightwtvar; run; • Useful “options1”: first • Useful “options2”: overid

  35. Software Considerations • For SPSS users: 2SLS • Basic command: 2sls y with x w / instruments z w / constant. • For point-and-click aficionados • Analyze → Regression → Two-Stage Least Squares • DEPENDENT, EXPLANATORY, and INSTRUMENTAL

  36. Software Considerations • For Limdep users: 2SLS • Basic command: 2SLS ; Lhs = y ; Rhs = one, x, w ; Inst = one, z, w ; Wts = wtvar ; Dfc $

  37. Application: Adolescent Work and Delinquent Behavior • Prior research shows a positive correlation between teenage work and delinquency • Reasons to suspect serious endogeneity bias • 2nd wave of the NLSY97 (N = 8,368) • Y = 1 if committed delinquent act (31.9%) • X = 1 if worked in a formal job (52.6%) • Z1 = 1 if child labor law allows 40+ hours (14.2%) • Z2 = 1 if no child labor restriction in place (39.6%)

  38. Regression Model Ignoring Endogeneity . reg pcrime work if nomiss==1 & wave==2 Source | SS df MS Number of obs = 8368 -------------+------------------------------ F( 1, 8366) = 6.33 Model | 1.37395379 1 1.37395379 Prob > F = 0.0119 Residual | 1815.97786 8366 .217066443 R-squared = 0.0008 -------------+------------------------------ Adj R-squared = 0.0006 Total | 1817.35182 8367 .217204711 Root MSE = .4659 ------------------------------------------------------------------------------ pcrime | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- work | .0256633 .0102005 2.52 0.012 .0056677 .0456588 _cons | .3053242 .0074009 41.26 0.000 .2908167 .3198318 ------------------------------------------------------------------------------ • Teenage workers significantly more delinquent • Modest effect but consistent with prior research

  39. First-Stage Model . reg work law40 nolaw if nomiss==1 & wave==2 Source | SS df MS Number of obs = 8368 -------------+------------------------------ F( 2, 8365) = 626.64 Model | 271.829722 2 135.914861 Prob > F = 0.0000 Residual | 1814.33364 8365 .216895832 R-squared = 0.1303 -------------+------------------------------ Adj R-squared = 0.1301 Total | 2086.16336 8367 .249332301 Root MSE = .46572 ------------------------------------------------------------------------------ work | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- law40 | .0688902 .0154383 4.46 0.000 .0386274 .099153 nolaw | .3818684 .0110273 34.63 0.000 .3602521 .4034847 _cons | .3655636 .0074883 48.82 0.000 .3508847 .3802425 ------------------------------------------------------------------------------ • State child labor laws affect probability of work • This is a really strong first stage (F, R2)

  40. Two-Stage Least Squares Model . ivreg pcrime (work = law40 nolaw) if nomiss==1 & wave==2 Instrumental variables (2SLS) regression Source | SS df MS Number of obs = 8368 -------------+------------------------------ F( 1, 8366) = 6.86 Model | -19.5287923 1 -19.5287923 Prob > F = 0.0088 Residual | 1836.88061 8366 .219564978 R-squared = . -------------+------------------------------ Adj R-squared = . Total | 1817.35182 8367 .217204711 Root MSE = .46858 ------------------------------------------------------------------------------ pcrime | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- work | -.0744352 .0284206 -2.62 0.009 -.1301466 -.0187238 _cons | .3580171 .0158135 22.64 0.000 .3270187 .3890155 ------------------------------------------------------------------------------ Instrumented: work Instruments: law40 nolaw ------------------------------------------------------------------------------

  41. What Do the Models Suggest Thus Far? • Completely different conclusions! • OLS = Teenage work is criminogenic (b = +.026) • Delinquency risk increases by 8.5 percent (base = .305) • 2SLS = Teenage work is prophylactic (b = –.074) • Delinquency risk decreases by 20.7 percent (base = .358) • Which model should we believe? • We still have some additional diagnostic work to do to evaluate the 2SLS model • Overidentification test, Hausman test

  42. Regression-Based Overidentification Test . reg IVresid law40 nolaw if nomiss==1 & wave==2 Source | SS df MS Number of obs = 8368 -------------+------------------------------ F( 2, 8365) = 0.25 Model | .111648085 2 .055824043 Prob > F = 0.7755 Residual | 1836.76895 8365 .219577878 R-squared = 0.0001 -------------+------------------------------ Adj R-squared = -0.0002 Total | 1836.8806 8367 .219538735 Root MSE = .46859 ------------------------------------------------------------------------------ IVresid | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- law40 | .010988 .0155334 0.71 0.479 -.0194613 .0414374 nolaw | .0016436 .0110953 0.15 0.882 -.020106 .0233931 _cons | -.0022127 .0075344 -0.29 0.769 -.0169821 .0125567 ------------------------------------------------------------------------------ • Overidentification test = 8,368 × .0001 = .8368 ~ χ2(1)

  43. Overidentification Test from the Software . overid Tests of overidentifying restrictions: Sargan N*R-sq test 0.509 Chi-sq(1) P-value = 0.4757 Basmann test 0.508 Chi-sq(1) P-value = 0.4758 • IV’s jointly pass the exogeneity requirement • Notice that -overid- provides a global test, whereas the regression-based approach allows you to test the IV’s jointly as well as individually

  44. Durbin-Wu-Hausman (DWH) Test Estimated by Hand • Summary coefficients • OLS model: b = +.026, s.e. = .010 • 2SLS model: b = –.074, s.e. = .028 • Notice the size of the 2SLS standard error • DWH = (–.074 – .026) / √(.0282 – .0102) ≈ –3.82 • CONCLUSION: Least squares estimate of the “work effect” is biased and inconsistent • The 2SLS estimate is preferred

  45. Regression-Based DWH Test . reg pcrime work FSresid if nomiss==1 & wave==2 Source | SS df MS Number of obs = 8368 -------------+------------------------------ F( 2, 8365) = 10.40 Model | 4.50567523 2 2.25283761 Prob > F = 0.0000 Residual | 1812.84614 8365 .216718009 R-squared = 0.0025 -------------+------------------------------ Adj R-squared = 0.0022 Total | 1817.35182 8367 .217204711 Root MSE = .46553 ------------------------------------------------------------------------------ pcrime | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- work | -.0744352 .0282357 -2.64 0.008 -.1297842 -.0190862 FSresid | .1150956 .0302771 3.80 0.000 .0557449 .1744462 _cons | .3580171 .0157106 22.79 0.000 .3272204 .3888139 ------------------------------------------------------------------------------ • Coeff. on work is bIV, while t-test on FSresid is DWH • Standard error for work is underestimated, though

  46. Or Just Let the Software Give You the DWH Test . ivendog Tests of endogeneity of: work H0: Regressor is exogenous Wu-Hausman F test: 14.45067 F(1,8365) P-value = 0.00014 Durbin-Wu-Hausman chi-sq test: 14.43093 Chi-sq(1) P-value = 0.00015 • Notice that -ivendog- provides a chi-square test for DWH, but the z-test that we computed by hand is easily recovered • √(χ2) = z  √(14.43) = 3.80

  47. Alternative Specifications for the Work-Delinquency Association • IV probit model • Without IV’s: b = +.072 (s.e. = .029) • With IV’s: b = –.207 (s.e. = .078) • Continuous work hours • Without IV’s: b = +.0015 (s.e. = .0003) • With IV’s: b = –.0024 (s.e. = .0009) • Indicator for “intensive” work (>20 hours) • Without IV’s: b = +.043 (s.e. = .012) • With IV’s: b = –.095 (s.e. = .036)

  48. Alternative Specifications for the Work-Delinquency Association • Control variables = gender, race, child, dropout, family structure, family size, urbanicity, dwelling, school suspension, unemployment rate, mobility • Binary work status • Without IV’s: b = +.013 (s.e. = .010) • With IV’s: b = –.061 (s.e. = .029) • Continuous work hours • Without IV’s: b = +.0007 (s.e. = .0003) • With IV’s: b = –.0023 (s.e. = .0010) • Intensive work indicator • Without IV’s: b = +.020 (s.e. = .012) • With IV’s: b = –.085 (s.e. = .040)

  49. So Where Do We Stand with the Work-Delinquency Question? • Are child labor laws correlated with work? • YES = first-stage F is large • Are child labor laws good IV’s? • YES = overidentification test is not rejected • Is teenage work endogenous? • YES = Hausman test is rejected • Prior research findings that teenage work is criminogenic are selection artifacts

  50. Stata Commands for the Foregoing Example • Regression model ignoring endogeneity: reg y x w • First-stage regression model: reg x z1 z2 w • With controls and multiple IV’s, test relevance: test z1 z2 • 2SLS regression model: ivreg y (x = z1 z2) w

More Related