When can School Inputs Improve Test Scores Education Finance and Decentralization

1. When can School Inputs Improve Test ScoresEducation Finance and Decentralization Jishnu Das (DECRG) James Habyarimana (Harvard University) Stefan Dercon (Oxford University) Pramila Krishnan (Cambridge University)

2. Motivation • Educational attainment key to human development and growth • Efficiency Reasons • More educated countries grow faster • More educated individuals have higher wages • More educated individuals have better outcomes for children • Equity Reasons • Accidents of birth and geography should not inhibit “ability to aspire” • Provide equal opportunities for all • Moral and Ethical Reasons • Education is an inalienable human right • Explicit aim of MDGs/PRSPs (59% of countries have minimal literacy aims)

3. How can Educational Attainment be improved? • Critical to assess inputs that result in greater educational achievement • Should we spend money on teachers (?), textbooks (?), teaching materials ? • Public Expenditure Tracking Surveys • Have assessed one important bottleneck to improved attainments: Does money provided by central budget office reach final recipients? (and if not, why not) {Ablo and Reinikka, Reinikka and Svensson, Das and others 2003 for Zambia) • Here: Even if resources reach, do they help? (and if not, why not?)

4. Existing Approaches • To assess what inputs affect educational attainment use “Educational Production Functions” • Idea: Educational Attainment = function (school inputs) • Examples • Test Scores = function (# textbooks) {Cross-Section} • Change in test scores = function (# textbooks) {Value-Added} • Also studies that examine impact of households on educational attainment • Examples • Cognitive Achievement = f(Income of Household) {Alderman and others, 1997} • School Enrollment = f(Household Labor Demand) {Jacoby and Skoufias

5. This Study (What is new) ? • Explicitly recognize that household responses will alter the manner in which school inputs affect test scores {Example} • Define: • PRODUCTION FUNCTION PARAMETER: Impact of school inputs on achievement if households do not/cannot respond. • POLICY EFFECT: Impact of school inputs on achievement if households do respond • In general PRODUCTION FUNCTION PARAMETER  POLICY EFFECT (Example: Textbooks) • Why is it important to evaluate both? • Policy: Critical to understand why certain inputs do/do not affect achievement for policy design (Example)

6. Theory • Case 1: Change in Test-Scores: Understanding Gains • PROPOSITION: If school inputs crowd out household inputs, then effect of unanticipated inputs > effect of anticipated inputs • Does this mean we should provide school inputs randomly? {long run impact} • NO! • Estimate on Anticipated: POLICY EFFECT • Estimate on Unanticipated: PRODUCTION FUNCTION PARAMETER

7. Data • Zambia • Survey of 182 schools (budget, spending, receipts, infrastructure) • Survey 541 households in remote areas to eliminate school choice (information on child-level educational expenditures, education of parents, demographics) • Test 2,600 children in 2001 and 2002 with same test • Also questionnaires on household assets of all children tested

8. Policy Environment (1) • Test proposition with non salary cash grants received by school • Two different sources of such grants • Legislated rule $600 to each school irrespective of enrollment: ANTICIPATED FUNDS • Discretionary Grants from District over and above $600: UNANTICIPATED FUNDS • Some characteristics • Anticipated funds reached 90% of all schools; remaining due to delay (confirmed later) • Unanticipated funds not “targeted” in any specific manner (Table 1 and Table 2)

9. Policy Environment (2) • Households very involved in children’s education • 60% attended last Annual General Meeting • 58% voted in Parent Teacher Association meeting • 60% reported home visits by teachers regarding child performance • Thus very likely • Households know about funding received in school • Households know how funding was spent in school

10. Econometric Tests (1) • Do Anticipated Grants crowd-out household educational expenditure? • Test 1: Use “Funding Received at time of survey” • Test 2: Use “Legislated Funds” • Idea: If substitutes, • If “truly anticipated” {Why?}

11. Crowding-Out of Household Expenditures • Ocular Impact Test • Regression Results • Main Result: Large and significant crowding out of household expenditures with respect to anticipated funds • (elasticity = 0.5; percentage terms 81% at mean household expenditure and mean anticipated funds • No evidence of crowding-out of unanticipated funds

12. Econometric Tests (2) • Effect on gain in learning • PROPOSITION

13. Effect on Test-Scores • Ocular Impact Test • Regression Results • Robustness (Instrumental Variables) • Comparison of anticipated and unanticipated funds

14. Effect on Test-Scores (2) • Main Result: Unanticipated Funds had a large and significant impact on test-scores (more so in English) • Anticipated Funds had no impact on test-scores for either English or Math • Confirming: Production Function Parameter > Policy Effect

15. Summary (1) • Households DO substitute own resources for anticipated school inputs • They adjust their own spending to the anticipated funding • They do NOT adjust to unanticipated funding • IF TEST: • Joint Hypothesis (`Policy Effect’): Funding matters AND households do not substitute THEN REJECT • Single Hypothesis (`Production Function Parameter’): Funding matters in the production function: CANNOT REJECT

16. Summary (2) : The View from Below • How plausible are these results? • Problem 1: Are households constrained in their responses? • Basic problem in schools teaching quality and infrastructure {main problems detailed by head-teachers} • Schools could not improve teaching quality {Thin Markets} • Schools could not improve infrastructure {Complementarities} • Ultimately forced to spend on materials that households could buy anyway • What did households think? • Problem 2: Are households able to gauge the performance of their children? • Matched data suggests yes

17. Policy Implications • Separation of Domains: Construction of “Spheres of Influence” • Domain of the Household vs. Domain of the School • For inputs where markets are functioning, leave provision to the household • What about the poor? Perhaps give the money directly, if we think credit constraints are the main problem • For inputs where markets are likely to be incomplete/imperfect, provide through the school • Companion work: Teachers matter, teacher absenteeism matters • Why? Problems with design of teacher contracts, imperfect markets.

18. `Remote’ Schools (HH Sample) Category Variable Urban Rural Percentage of schools who received anticipated 89.4 89.3 85.7 funds at time of survey (3.7) (2.9) (5.9) Percentage of schools who received 23.1 24.8 14.2 unanticipat ed funds (5.2) (4.0) (5.9) Anticipated amount received (log Kwacha per 7.66 8.67 8.93 Cash - Grant pupil) (0.41) (0.60) (0.54) Characteristics Unanticipated amount (log Kwacha per pupil) 7.22 7.93 9.84 (2.31) (2.58) (2.77) Table 1: Funding Characteristics Return

19. Received discretionary No discretionary Significant Category Variable funds received funds difference? School Asset Index - 0.03 .09 (0.79) (.77) Total Enrollment in School 862 981 School Distance to District Office (% 48.1% 60.4% characteristics within 5 KM) Distance to Provincial Office (% 20.7% 18.6% within 5 KM) English Scores in 2001 - 0.045 - 0.05 Performance in (.48) (.52) 2001 Mathematics Scores in 2001 - 0.012 - . 069 examinations (0.47) (0.42) Table 2: Targeted Funding? NO NO NO NO NO NO Return

20. (1) (2) (3) (4) (5) (6) Base Hypothesis Test of Weak Regression: 2: Tobit Test of Exogeneity: Base Tobit with Hypothesis with Weak Tobit with Random Random random Regression: 2: Tobit Exogeneity: Effects Ef fects Tobit effects Tobit Specification Log -0.417 -0.414 Anticipated [0.147]** [0.180]* Funds (Received at time of survey) Log -0.567 -0.572 -0.759 -0.763 Anticipated [0.142]** [0.160]** [0.298]* [0.332]* Funds (Legislated) Log -0.079 -0.079 -0.077 -0.077 -0.177 -0.175 Unanticipated [0.058] [0.065] [0.056] [0.062] [0.064]** [0.078]* Funds Residual From 0.235 0.232 “Selection” [0.343] [0.383] Equation Controls Yes Yes Yes Yes Yes Yes Constant 10.214 10.264 11.834 11.860 9.693 9.673 [1.320]** [1.486]** [2.722]** [3.034]** [1.589]** [1.940]** Observations 1410 1410 1410 1410 1410 1410 Regression Results: Substitution Return

21. Hurdle IV Hurdle IV: Expected Expected Comparison: Comparison Rule Funds Rule Funds (OLS, (OLS, (English) (Math) English) Math) Hurdle 0.128 0.101 0.082 0.039 instrumented log [0.052]* [0.033]* [0.031]* [0.020] unanticipated grants Hurdle -0.013 -0.009 -0.006 -0.002 Instrumented log [0.006]* [0.003]** [0.003]* [0.002] unanticipated grants squared Log of -0.110 0.038 -0.012 0.024 anticipated [0.045]* [0.047] [0.020] [0.019] grants Observations 164 164 164 164 R-squared 0.18 0.04 0.15 0.05 Regression Results: Test-Scores Return

22. Test Scores Gain (2002 – 2001) Hh Flows 2001 2002 Time Changing Test Scores

23. Test Scores Gain (2002 – 2001) Hh Flows Hh Flows School Flows 2001 2002 Time Impact of Anticipated Flows on Test Scores

24. Test Scores Excess Gain above target Gain (2002 – 2001) Hh Flows School Flows 2001 2002 Time Impact of Unanticipated Flows on Test Scores Return

25. Test Scores Hh Flows Hh Flows Hh Flows School Flows School Flows 2001 2002 2003 Time Long Term Effects of Anticipated Flows

26. Excess Gain above target Hh Flows Hh Flows Hh Flows School Flows School Flows 2001 2002 2003 Time Long Term Effects of Unanticipated Flows Test Scores Return

27. Household Substitution: Ocular Impact Return

28. Ocular Impact: Effect on Test Scores Return

29. Households’ Thoughts Return

30. Household’s Ability to gauge performance Return