1 / 8

Pay for performance and impact evaluation design Practical lessons from OECD review

Learn practical lessons from the OECD review on designing pay-for-performance and impact evaluations. Explore quantitative, experimental, and non-experimental methodologies, including randomized control trials. Understand the importance of internal and external validity while considering the timeline, civil society involvement, and implementation challenges. Discover how to create impact models, translate them into measurable indicators, and plan resources effectively for successful evaluations. Emphasizing the need for qualitative assessments, this review sheds light on evaluation practices in OECD countries, highlighting areas for improvement based on examples like the US Premier Hospital scheme and QOF.

oihane
Télécharger la présentation

Pay for performance and impact evaluation design Practical lessons from OECD review

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Pay for performance and impact evaluation design Practicallessonsfrom OECD review Y-Ling Chi, OECD

  2. Methods available • Quantitative methods • Experimental methods Golden standard • Non-experimental methodology • Quasi experimental methods • Qualitative methods

  3. Non-experimental/Quasi experimental • Quasi-experimental: Interrupted times series, regression discontinuity and score matching • Non experimental: before/after intervention or treatment/control afterwards

  4. Experimental methods = Randomized Control Trials (RCTs) • Easy to understand • InternalValidity • ExternalValidity BUT • Timeline (very long) • Strong civil society (hard to comply) • Hard to implement (need to have experts) • Expensive

  5. Treatment Treatment INTERVENTION GROUP control control t -0 intervention t +1 Evaluation: difference between control and treatment // difference in t-0 and t+1

  6. How to understand Impact Evaluation? • Design a proper impact model • Translate impact model into quantifiable indicators • Think about the design model • Timing has to articulate within the implementation process • Plan resources ahead (staff, implementation, analysis etc.) • Once you finish, need good qualitative evaluation

  7. Review from OECD countries • OECD countries don’t do well in terms of evaluation US Premier Hospitalschemeexample shows thatevaluationis not central once the prorgam has been implemented. QOF shows thatdespite the yearlycost -1 billion pounds-, investment for evaluationis marginal but not evenwellperformed.

More Related