Professor Graduate School of International Cooperation Studies Kobe University

1. Uganda Evaluation Week 2014Building Evaluation Capacities, Culture and Practices in UgandaHow Evidence Can Be Used for Better Public Services: Case of Education SectorKeiichi Ogawa, Ph.D. Professor Graduate School of International Cooperation Studies Kobe University

2. Outline of the Presentation 1. Introduction 2. What is Evidence for Policy, 3. Why do we need Evidence? 4. Evidence from Uganda’s Education System -Part I: Basic Statistics (Facts & Figures) -Part II: Effectiveness Analysis on Primary Education in Uganda -Part III: Efficiency Analysis in Lower Secondary Education in Uganda 5. Evidence from International Perspective • Cost-Effectiveness Analysis

3. 1. What is Evidence for Policy? (1/2) • A discourse or methods that inform the policy process, rather than aim to directly influence the eventual goals of the policy. • Requires a more rational, rigorous and systematic approach. • Policy which is based on systematic evidence is seen to produce better outcomes.

4. 2. What is Evidence for Policy? (2/2/) • Better utilization of evidence in policy and practice can help save lives, reduce poverty and improvedevelopment performance in developing countries. Some examples, Government of Uganda has implemented systematic program of UPE, informed by the results of annual national school censuses and Head Count; • Access increased from 3 Million in 1996 to 8.4 Million 2010; • No. of schools increased from 12,500 in 2000 to 19,797 in 2009 • GER reduced from 128% in 2000 to 114% in 2010; • Teachers doubled (74,000 in 1995 to 145,302 in 2009)

5. 3. Why do we need Evidence? Shaxson (2005) argues that we need evidence to: • Understand the policy environment and how it’s changing; • Appraise the likely effects of policy changes, choice to different policy options and assessing their impacts; • Links between strategic direction, intended outcomes and policy objectives; • Determine what we need to do to meet our strategic goals or intermediate objectives; • Influence others so that they help us achieve our policy goals and take them through to delivery; • Communicate the quality (breadth and depth) of our evidence base to meet the government agenda.

6. EVIDENCE FROM UGANDA’S EDUCATION SYSTEM

7. A) Background • In the past two decades, Uganda embarked on major education reforms (e.g. UPE in 1997 and USE in 2007) which have largely focused on equalizing educational opportunities as part of a broad development strategy to achieve EFA and MDGs. • Significant investments have been made towards inputs in the areas of: Curricula, teaching/ learning processes, school inspection and support supervision teachers’ accommodation, absenteeism, school facilities and infrastructure. These have paid off with an increase in enrolment figures over time. The challenge however that still stands is whether the millions of children in school for whom these investments have been made are learning. • Education continues to take a large share of the budget but is also a sub-sector where the evidence of waste is evident and rampant (MoFPED 2012). One key area of current debate has been inability and poor quality of school Management to use the available resources efficiently (MoES 2010).

8. B) Structure of Uganda’s Education System

9. PART 1:BASIC STATISTICS – FACT & FIGURES

10. A) Evidence from UPE Program – Access Issues Trends in Primary School Enrollment by Gender

11. B) Evidence from UPE Program – Quality Issues Percent of P3 and P6 pupils rated proficient in Literacy

12. C) Evidence from UPE Program – Quality Issues Percent of P3 and P6 pupils rated proficient in Numeracy

13. D) Evidence from UPE Program – Internal Efficiency Progression Rates between Classes for 1997-2004 Cohorts

14. E) Uganda Compared to SSA Countries - SACMEQ

15. F) Compared to SSA Countries - SACMEQ

16. G) Evidence from USE Program –Access Issues USE Enrollment trends over the years, 2007-2011

17. H) Evidence from USE Program –Quality Issues Completion rate at S4

18. I) Evidence from USE Program –Quality Issues Transition rates to S5

19. Part II Effectiveness Analysis on Primary Education in Uganda

20. A) Research Issues/Questions To what extent does the family factors (e.g. pupil age & sex, language spoken at home, pupil staying with parents, ensuring pupil’s daily attendance, the number of books at home, pupil helped with homework at home, parents education, home-possessions and having electricity etc.) correlate with pupil’s academic achievement? To what extent do the school factors (e.g. subject teachers appreciating the conditions of classroom, teachers qualification and years of experience, a pupil repeating grade six, pupil borrowing books from the library, pupil given home work, school head and teacher’s qualifications, number of years of service (experience), school type influence pupils’ academic achievement? To what extent does the community variables (e.g. community’s regular meetings with the teachers and school heads, paying salary for additional staff and topping-up teachers’ salaries, location (rural and urban) of the school influence on the pupil’s academic achievements?

21. B) Methods and Data Use an Education Production function as proposed by Glewwe and Kremer (2005) Where Tis is the score of student i from school s, Hj = is a vector of home context characteristics, Sk =is a vector of school system context characteristics, Cl= is a vector of community characteristics Data Issues Southern African Consortium for monitoring Education Quality (SAQMEQ II & III). It focused on Uganda because it is one of the fifteen countries that participated in the SACMEQ survey.

22. C) Key Findings

23. D) Implications for Policy The community meeting teacher variable emerges as a significant variable explaining performance. Government should organize monthly or quarterly sensitization meetings with communities so that they know the value of education to children. Rural parents may need extra form of financial support to develop schools and personal income. Central government should increase the equalization funds currently being paid to disadvantaged local governments and support schools started by communities The ministry of education and sports could find ways of dealing with grade repetition without lowering the standards of achievement by employing (Brophy 2006) strategies such as – early intervention, collaboration with parents, and supplementary instructions. Central government should develop guidelines and put in place a policy for all local governments and school head teachers to regularly sensitize parents to provide the basic needs of their pupils and provide help for them at home with their studies.

24. Part III Efficiency Analysis in Lower Secondary Education in Uganda

25. A) Research Issues/Questions Question 1: What is the current technical efficiency level of schools in Uganda? Question 2: How do school factors (e.g. school size, proportion of female students, heterogeneity of students body, number of career guidance services and number of female stances, school ownership, USE status, co-educational type and boarding type & location) affect its technical efficiency? Question 3: How do students and family socio-economic (e.g. student’s age, per student family expenditure on education and population density) factors affect school technical efficiency?

26. B) Methods & Data Objective 1: Estimating School Technical Efficiency indices using non-parametric technique - Data Envelopment Analysis (DEA) Model Objective 2-3: Use of Tobit Model are efficiency scores from DEA analysis & εi is the error term school size (SIZ), boarding status (SBT), co-educational (SCT), proportion of female students (PFS), female stance ratio (FSR), heterogeneity of students’ body (HET),career guidance activities (CGA), school USE status (USE), ownership (OWN), location (LOC), students’ age (AGE), family expense on education (HHE), population density (POP) and regional dummies for eastern (EDV), northern (NDV) and western (MDV); Βj are vectors of parameters to be estimated

27. C) Key Results – Objective 1

28. Summary of Results – Objective 1 • Of 283 schools, 70 (24.7%) of them are efficient, and the rest (75.3%) are far from the efficiency frontier. • 52 (28.4%) out of 183 are efficient in rural compared to 18 (18.0%) of the 100 urban schools. • Of 200 public schools, 49 (24.5%) of them are efficient compared to only 21 (25.3%) out of 83 private schools. • 38 (20.4%) of the 186 USE are efficient compared to 32 (33.0%) of the 97 non-USE schools. • There is significant difference (at 1%) between productive efficiencies of schools under USE policy and non-USE

29. D) Key Results – Objectives 2 & 3

30. Summary of Results – Objectives 2& 3 • School factors that demonstrate significant; Positive effect : • School size, • Co-educational status, • Proportion of female students and their stances, • Number of carrier guidance services. • Location (regional) effects Negative effect : • Heterogeneity of students body • Student & Family factors: Positive effect:Family expenditure, regional effects; Negative effect:Students’ age group effects

31. E) Implications for Policy • The introduction of USE policy increased schools’ enrolments that constrained resources creating harsh conditions for schools to operate. It was/is considered “transition” period engulfed with management and financing challenges. • Expenditures on education reflect increased investment in students’ learning and also financial mark-up to schools. So, government may design funding mechanism to public schools to narrow the financial economic differences between private and public schools. • Most schools attract children from poor families and charge low fees. Funding constraints suffocates other basics. • In large schools, high achievement are possible as long as resources are available and proper use of them is assured and sustainable. The education policies should emphasize improving school size as long as class sizes are maintained standard/reasonable.

32. EVIDENCE FROM INTERNATIONAL PERSPECTIVECOST-EFFECTIVENESS ANALYSIS:

33. 6.Cost-Effectiveness Analysis of Education ProjectCost-Effectiveness Analysis of Education Project Evaluation with Experimental Data: An International Comparison Approximate cost for increasing years of schooling by 1 year (U.S. dollars) Microfinance (Guatemala) Cash transfer (Mexico) Education voucher (Colombia) Scholarship (Kenya) School construction (Indonesia) Increase in provision of assistant teachers (India) Introduction of teacher incentives (India) School feeding (Bangladesh) Deworming drugs (India) School feeding (Kenya) Free of recurrent education costs (Kenya) Deworming drugs (Kenya) (Upper axis) (Upper axis) (Upper axis) (U.S. dollars) Source: Ogawa, Nakamuro, & Hoshino (2009)

34. 6.Cost-Effectiveness Analysis of Education ProjectCost-Effectiveness Analysis of Education Project Evaluation with Experimental Data: An International Comparison Approximate cost for increasing a test score by 0.1 standard deviation Education voucher (Colombia) Reducing class-size (Honduras) Early childhood education program (Philippines) Computer-aided instruction (India) Textbook (Kenya) Teacher incentives (India) Teacher incentives (Kenya) Scholarship (Kenya) Classroom renovation (Ghana) Capitation grant (Uganda) School furniture (Philippines) Supplementary lesson implementation (India) Teacher training (Honduras) Workbook for students (Philippines) Black board (Ghana) (U.S. dollars) Source: Ogawa, Nakamuro, & Hoshino (2009)

35. Thank You