150 likes | 155 Vues
Training Seminar for Evaluation Plan Committee Members, Bucharest, 26/2/2007. Introductory Presentation by David Hegarty David Hegarty Phone: +35312986379 EVALCON Mobile: +353879819905 Evaluation and Economic Consultancy Services Email: david.hegarty@evalcon.eu. Presentation Structure.
E N D
Training Seminar for Evaluation Plan Committee Members,Bucharest, 26/2/2007 Introductory Presentation by David Hegarty David Hegarty Phone: +35312986379 EVALCON Mobile: +353879819905 Evaluation and Economic Consultancy Services Email: david.hegarty@evalcon.eu
Presentation Structure • Monitoring and relationship with evaluation • What is evaluation and why is it done? • Evaluation criteria • Evaluation cycle • Conclusion
What is monitoring? • An ongoing, continuous, systematic process • Management function • Examines or checks programme coverage and delivery • Some monitoring questions • How much has been spent? • What did we get for it? • Who benefited? • Are we on track? • Uses financial and performance indicator data
Monitoring and Evaluation • Monitoring and evaluation are linked processes • But monitoring is ongoing • Evaluation is discrete • Evaluators use monitoring information • Indicators • Monitoring acts as an early warning system • Highlights problems or areas that require evaluation
What is evaluation? • “Evaluation” is a common, everyday activity • People evaluate films, restaurants etc • Firms evaluate investments • “Evaluation is an elastic word that stretches to cover judgements of many kinds” (Weiss) • Public sphere • A tool used in management of public policies, programmes and projects • Other tools are monitoring and audit
Defining Evaluation • Some definitions: • “A judgement of interventions according to the results, impacts and needs they aim to satisfy” (EU Commission) • “A critical and detached look at the objectives and how they are being met” (UK Treasury) • Involves judgement on basis of criteria • More comprehensive than monitoring • Applies to policies, programmes and projects
Why evaluate? • Evaluation of EU programmes is mandatory • 3 main overt purposes in public sector context • Planning: to assist in the allocation of resources (ex-ante) • Implementation: To improve programme management (interim evaluation) • Accountability: To address accountability concerns (ex-post) • But evaluation sometimes used for other purposes • Justify decisions already made • Postpone decisions • Public relations • Compliance • “A rational exercise often undertaken for non-rational reasons” (Weiss)
Purpose of Evaluation • Overarching learning purpose “To learn through systematic enquiry how to better design, implement and deliver public programmes and policies” (EU Evalsed Guide)
Evaluation Types • Summative evaluation • Accountability focus • What has been achieved? • What's the value of a programme? • Formative evaluation • Development or learning focus • How can we improve performance and delivery of programme? • “When the cook tastes the soup that’s formative evaluation. When the guest tastes it, that’s summative evaluation” • Both relevant and useful to public sector
Evaluation Criteria • What's the basis for evaluation judgements? • 5 main criteria • Rationale • Relevance • Effectiveness • Efficiency • Impact/Sustainability • Effectiveness issue is where monitoring and evaluation meet
Evaluation Criteria Needs Problems Issues Impacts Society Economy Environment Results Intervention Programme Objectives Inputs Outputs Efficiency Rationale & Relevance Effectiveness Evaluation Impact/Sustainability
The EU Evaluation Cycle • Before (ex-ante evaluation) • Completed in Romania for each OP • General aim: improve allocation of resources and programme design • During (ongoing evaluation) • External developments and their implications? • Is the programme meeting its objectives? • Can we improve programme management? • Required under General Regulation • Managing authority responsibility • Evaluation plan • After (ex-post evaluation) • What has been achieved? • What difference did it make? • EU Commission responsibility
Policy Review Policy Formulation Programme Conclusions Programme Design Ex-post Evaluation Ex-ante Evaluation Interim Evaluation Programme Implementation Policy Delivery Policy, Programme & Evaluation Cycles Source: The Guide
Summary/Concluding Points • Relationship between monitoring and evaluation • Purpose of evaluation …. • Need to have an evaluation framework • Evaluation questions or criteria • Focus varies over policy/programme cycle