1 / 26

CONSISTENCY AND RESULTS EVALUATION IN MEXICO

CONSISTENCY AND RESULTS EVALUATION IN MEXICO. Gonzalo Hernández Licona. SEMPTEMBER 2008. The Context: Performance Evaluation System (SED) Consistency and Results Evaluation. Results-Based M&E System. Planning and Indicators. Evaluation. Annual Evaluation Program. Consistency and

okalani
Télécharger la présentation

CONSISTENCY AND RESULTS EVALUATION IN MEXICO

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CONSISTENCY AND RESULTS EVALUATION IN MEXICO Gonzalo Hernández Licona SEMPTEMBER 2008

  2. The Context: Performance Evaluation System (SED) Consistency and ResultsEvaluation

  3. Results-Based M&E System Planning and Indicators Evaluation Annual Evaluation Program Consistency and Results Evaluation Strategic goals: Ministries Results Impact Evaluation Programs aligned to Strategic goals Process Evaluation Program indicators (Results): Logical Framework Performance Evaluation Strategic Evaluation Follow up on evaluation results

  4. Performance EvaluationSystem (SED). Systematic information about the program’s performance Beginning in 2009: I I I E E I I I I

  5. SED. Indicators 1 % Budget Spent 100 80 60 40 20 0 1er trimester 2o trimester 3er trimester 4to trimester 2009 (Scholarship Program)

  6. SED. Indicators 2 Number of scholarships granted 250,000 Goal 200,000 150,000 100,000 50,000 0 1er trimester 2o trimester 3er trimester 4to trimester 2009 (Scholarship Program)

  7. SED. Indicators 3 Coverage Indicator 100 90 80 70 60 50 40 30 20 10 0 2012 2014 2016 2018 2020 2022 2006 2008 2010 2024 (Scholarship Program)

  8. SED. Evaluation 1 Impact in educational years 15 14 Beneficiaries 13 12 Control Group 11 10 9 8 7 6 0 2011 2012 2013 2014 2015 2016 2008 2009 2017 2010 (Scholarship Program)

  9. Consistency and Results Evaluation. What is it? The main goal: to what extent social programs are oriented toward results, based on their own documentation and processes? It’s a diagnosis about the institutional and management capacity of the programs to achieve results. The idea was to have a homogeneous instrument, with measurable answers, in order to compare performance among programs. The evaluation took place in 2007-2008 for 119 programs. This evaluation is a baseline showing what type of tools (programs) the new Government has to achieve its goals. The next evaluation will take place in 3 years time.

  10. Consistency and Results Evaluation. Characteristics The evaluation has 100 questions about the following topics, for every program: Design, Strategic Planning, Target Population and Coverage, Operation, Perception of Beneficiaries and Final Results. The Design topic is mainly based on the Logical Framework. 80 of the questions have a binary answer (YES/ NO) The evaluations were done by external institutions (universities, consultants, experts), hired by the ministries. 7 of them hired by CONEVAL. The evaluation includes a Summary:Strengths, Challenges and Recommendations for every program.

  11. DESIGN Does de program identifies clearly the problem it’s trying to solve? Is it clear that the products or services produced by the program logically contribute to reduce the problem the program is tacking? STRATEGIC PLANNING Does the program have updated strategic plans for the short, middle and long-term? In the S-plan, does the final results are clearly established? TARGET POPULATION AND COVERAGE Does the program have a method to identify and quantify the potential and target population? Does the program have a coverage strategy for the short, middle and long-term? Consistency and Results Evaluation. Questions

  12. OPERATION Are there standard and adequate procedures for the selection of projects, beneficiaries? Are there documents showing that the procedures are done according to the rules of operation. PERCEPTION OF BENEFICIARIES Does the program have instruments to measure the satisfaction of the beneficiaries with the program? FINAL RESULTS Does the program systematically collect objective information about its Goal (Fin y Propósito)? Does the program have rigorous impact evaluations? Consistency and Results Evaluation. Questions

  13. Consistency and Results Evaluation. General Findings

  14. Consistency and Results Evaluation. General Findings

  15. Consistency and Results Evaluation. Findings per Program

  16. GOAL 5:Reduce extreme poverty and assure equal opportunities. MilkSupplyProgram (LICONSA) CORE: Equal Opportunities THEME: Poverty Alleviation EVALUATION / RESULTS

  17. GOAL 5:To reduce the extreme poverty and to assure equal opportunities. MilkSupplyProgram (LICONSA) CORE: Equal Opportunities THEME: Poverty Alleviation

  18. GOAL 5:To reduce the extreme poverty and to assure equal opportunities. MilkSupplyProgram (LICONSA) CORE: Equal Opportunities THEME: Poverty Alleviation

  19. GOAL 5:To reduce the extreme poverty and to assure equal opportunities. MilkSupplyProgram (LICONSA) CORE: Equal Opportunities THEME: Poverty Alleviation

  20. GOAL 5:Reduce extreme poverty and assure equal opportunities. Rural HousingProgram CORE: Equal Opportunities THEME: Poverty Alleviation EVALUATION / RESULTS

  21. Consistency and Results Evaluation. What did we do? EVALUATION: ACCOUNTABILITY AND PERFORMANCE IMPROVEMENT The evaluations were sent to Congress and the Ministry of Finance in April The complete evaluations can be found in the web page (CONEVAL and Ministries). The scores were sent to Presidency in order to take better budget decisions for 2009. We don’t know the results yet… Nowadays, programs are analyzing the pertinent and feasible recommendations given in the Consistency and Results Evaluation, and they are doing a Work Plan to improve their performance. This Work Plan will be public.

  22. Summary The ECR ispart of the SED. It’sperhapsone of thefirstinstruments of the SED. Wewereabletoget a homogeneouspicture of 119 programs. We’reimprovingtheinstrument. Theinformationwillbeusedtoimprovetheprogram Theinformationis in the web. ThePresidentwantedtheinformationforbudgetpurposes, butwedon’tknowifthiswasthe case. TheWork Plan toimprovetheprogram, willbe a bridge connectingAccountability and Performance improving.

  23. 2009

  24. SED. Valuable information menu I I I E E Performance Evaluation (Annual Summary) I I I I

  25. Annual Performance Evaluation 4 pages report, summarizing the information contained in the SED (to be read by Congress): The Report will have: Name and goal Budget (last 3 years) Results (shown in SED; through indicators or Evaluations) Coverage (shown in SED) Financial indicators (shown in SED) …. Recommendations (1 or 2) If there is no information about some of these subjects, it must be reported

  26. Consistency and Results Evaluation. Challenges Huge variance in the quality of external evaluators. CONEVAL re-evaluated 47 evaluations. Questions: A lot of questions; some duplications. We are improving the instrument. The evaluation was highly linked to the Logical Framework. The programs (and evaluators) had just started to know this methodology.

More Related