Good Practice Government Systems for M&E

1. Good Practice Government Systems for M&E The Cases of Chile and Colombia Presentation to a LAC Region Summer Seminar 9 August 2006 Keith Mackay Independent Evaluation Group (kmckay@worldbank.org)

2. What Does “Success” Look Like?-- Why Countries Want an M&E System • To support budget decision-making = performance-based budgeting • To support national and sectoral planning • To design policies and programs • To assist sector ministries / agencies in their management • To strengthen accountability relationships

3. Chile’s M&E System -- Architecture • Designed, managed and used by Hacienda • Developed incrementally, over past decade • Performance indicators (∑1,600) for all government programs (1994) • Government program evaluations (∑ 160) -- these are desk reviews (1996) • Rigorous impact evaluations (∑14) (2001) • Comprehensive Spending Reviews -- desk reviews of all programs in a functional area (2002)

4. Chile’s M&E System -- Strengths (1) • ‘Graduated’ approach to M&E • Evaluations conducted externally, in fully transparent process, and are highly credible • All M&E findings reported publicly and sent to Congress • M&E system closely linked to the information needs of Hacienda, especially for budget process • Performance information used to set performance targets for ministries -- these are largely met

5. Chile’s M&E System -- Strengths (2) • High utilization of M&E findings by Hacienda in the budget process and to impose management improvements on ministries / agencies (see Table)

6. Chile’s M&E System -- Challenges • Unevenness in quality of evaluations -- due to cost and time constraints • Chile probably not spending enough on evaluations • Low utilization -- low ‘ownership’ -- of Hacienda’s evaluations by sector ministries

7. Colombia’s M&E System (SINERGIA) -- Architecture • SINERGIA is managed by the Department of National Planning (DNP), with strong support from the Presidencia • On-line sub-system -- SIGOB -- for monitoring and reporting government progress vis-à-vis Presidential Goals • Ambitious agenda of impact evaluations (∑15 underway) • DNP provides technical assistance to a few ministries/agencies to develop M&E, and to municipalities to pilot SIGOB and performance-based budgeting

8. Colombia’s M&E System -- Strengths • Very high utilization of SIGOB by President for oversight of ministers and ministries -- via performance targets -- and for accountability, i.e. ‘social control’ • Rigorous impact evaluations conducted externally, and have high credibility • Collaborative approach between DNP and sector ministries/ agencies, and with municipalities • Performance budget reports; efforts to further strengthen performance budgeting • Efforts to engage with civil society

9. Colombia’s M&E System -- Challenges • Too high reliance on donor funding for SINERGIA --low level of government funding support • Insufficient reliance on M&E information to support national planning and budget decision-making -- this may now be changing • Weak coordination of M&E roles / functions within DNP and with central and sector ministries • SIGOB data quality perceived as low

10. How to Define a Government M&E System as Being “Good Practice” • Can be dangerous concept -- each country’s starting point and desired end-point are unique • What a “successful” M&E system is NOT: • complex set of laws, decrees, regulations • # performance indicators collected • # rigorous impact evaluations conducted • # rapid evaluations conducted • = these simply reflect the architecture of the system or are measures of M&E effort

11. How to Define a Government M&E System as Being “Good Practice” • Chile and Colombia have good-practice M&E systems because • the quality of their M&E work is generally reliable, and • in particular, the monitoring information and evaluation findings which they produce are used intensively • High utilization reflects strong demand for M&E, and can be viewed as a predictor of an M&E system’s sustainability

12. Lessons (1) • Lessons from Chile and Colombia are consistent with other countries’ experience • Key role of powerful champion of M&E • Opportunistic development of M&E systems, via continuous review and modification; non-linear development of the systems • Centrally-driven, by capable ministry • Incentives are key -- cultural change to strengthen demand, achieve high utilization • Avoid competing systems – Planning, Finance

13. Lessons (2) • Build reliable ministry data systems • Role of structural arrangements to ensure M&E objectivity and quality • Long-haul effort, requiring patience • Limitations of relying on laws, decrees, regulations • An M&E system can be built and operated are relatively low cost • Chile’s M&E system costs $0.75m p.a. • Colombia’s system costs about $2m p.a.

14. Useful Resources (1) • Chile:www.dipres.cl/fr_control.html • Colombia:www.dnp.gov.co/paginas_detalle.aspx?idp=266 • Ernesto May et al. (eds.), Towards the Institutionalization of Monitoring and Evaluation Systems in Latin America and the Caribbean, World Bank/IADB, 2006.http://web.worldbank.org/WBSITE/EXTERNAL/COUNTRIES/LACEXT/0,,contentMDK:20893139~pagePK:146736~piPK:146830~theSitePK:258554,00.html(Disponible en Español) • Keith Mackay, Institutionalization of Monitoring and Evaluation Systems to Improve Public Sector Management, Independent Evaluation Group, World Bank, 2006.www.worldbank.org/ieg/ecd/institutionalizing_me.html(Disponible en Español)

15. Useful Resources (2) • Ariel Zaltsman, Experience with Institutionalizing M&E Systems in Five Latin American Countries: Argentina, Chile, Colombia, Costa Rica and Uruguay, Independent Evaluation Group, World Bank, 2006.www.worldbank.org/ieg/ecd/experience_five_la.html • Fernando Rojas et al, Chile: Análisis del Programa de Evaluación del Gasto Publico, World Bank, 2005.http://iris37.worldbank.org/domdoc/PRD/Other/PRDDContainer.nsf/WB_ViewAttachments?ReadForm&ID=85256D2400766CC785257155005CB26B& • World Bank website on Building Government M&E Systems: www.worldbank.org/ieg/ecd/