Evaluating Interventions: Selected Cases by Professor Prosper Ngowi

1. The What, When, Why and How Questions in Evaluating Interventions: Selected Cases Professor Prosper Ngowi; pngowi2002@yahoo.com +255 754 653 740 1

2. Outline • Evaluation of various interventions in forms of projects and programmes • Selected key evaluation issues • What is normally being evaluated • When should evaluations be done • Why evaluations are done • Who should do evaluation • How can evaluations be done • Source of information: • Theoretical and practical hands-on involvement in over 20 evaluation assignments

3. Introduction & context • Evaluation as part of project cycle management • Selected evaluation assignments/cases • About 20 evaluation and related assignments • Actual evaluation of projects • Mid-term and end of project evaluations • Public, private as well as civil society organizations (CSOs) sectors projects

4. Selected Evaluation Projects • End-of-project evaluation of Bukombe Community Economic Strengthening Initiative (Heifer 2017) • Evaluation of the Impacts of BTC primary health project in Karagwe – May – June 2009 • Evaluation of the Impacts of the Regional Facilitating Agencies (RFAs) for TACAIDS in Mwanza and Kagera • Evaluation of the Impacts of DANIDA agricultural projects in Mbeya and Iringa – February 2004 • Evaluation of the Impacts of MKURABITA Programme for the governments of Norway and Tanzania, Sept 2005

5. Cases … • Evaluation of the Impacts of UNDP Volunteers Programme on HIV/AIDS in Kagera, Kigoma, Rukwa and Lindi: 2007 • Evaluation of the Impacts of Medicos Del Mundo Primary Health Project in Karatu and Ngorongoro – 2008 • Evaluation of KOICA Projects in Tanzania – September – October 2008 • Final Evaluation of Beekeeping Project in Rufiji, Kigoma and Kibondo for BTC (May – July 2010) • Final Evaluation of Kilombero Valley Ramsar Site Integrated Management Plan for BTC (December 2010 to January 2011) • Evaluation of the MCA-T – January to February 2013 • Evaluation of Water Sector Development Program (WSDP), 20113 • n

6. Cases … • Appraisal of DANIDA’s private sector development programme (February – March 2013) • Review of ILO Youth Employment Facility (YEF) in Tanzania, Kenya and Uganda (September 2013) • Evaluation of Binti Simama Imara Project for Social Action Trust Fund (SATF), Nov to Dec 2014 • Evaluation of CSOs Capacity Building/Support Programme for WWF; Dec 2014 to January 2015. • Evaluation of CARE International Pastoralist Programme: March – May 2016 • Evaluation of YEE) Project for Plan International (May 2017)

7. What was evaluated: The OECD evaluation criteria • Relevance: The extent to which intervention is suited to the priorities and policies of the target group, recipient and donor • Key relevance questions: • To what extent are the objectives of intervention still valid? • Are the activities and outputs of the intervention consistent with the overall goal and the attainment of its objectives? • Are the activities and outputs of the intervention consistent with the intended impacts and effects?

8. What is evaluated … • Effectiveness • A measure of the extent to which an intervention attains its objectivesKey effectiveness questions: • To what extent were the objectives achieved / are likely to be achieved? • What were the major factors influencing the achievement or non-achievement of the objectives?

9. What is evaluated … • Efficiency • Measures the outputs in relation to inputs • Assesses whether intervention uses the least costly resources possible in order to achieve the desired results • Compares alternative approaches to achieving the same output; Is about adopting the most efficient process • Key efficiency questions: • Were activities cost-efficient?; Objectives achieved on time? • Was the intervention implemented in the most efficient way compared to alternatives?

10. What is evaluated … • Impact: Is about: • Positive and negative changes produced by an intervention, directly or indirectly, intended or unintended • Impacts and effects resulting from the intervention (causality, contribution, attribution) • Key impact questions: • What has happened as a result of the intervention? • What real difference has the activity made to the beneficiaries? • How many people have been affected?

11. What is evaluated … • Sustainability • Is concerned with measuring whether benefits of an activity are likely to continue after project exit • Environmental and financial sustainable • Key sustainability questions: • To what extent did the benefits of intervention continue after project exit • What were the major factors which influenced the achievement or non-achievement of sustainability of the intervention?

12. Why evaluate? • Tracking record: • Improving remaining part of project (mid-term) • Informing similar projects and projects in similar environment • Learning

13. When to evaluate? • Mid-term • At project’s half life cycle • End/final/terminal • Just before project closure

14. Who can evaluate? • Internal evaluators • Can be biased, partial, subjective • External evaluators • Unbiased, impartial, new look/perspectives • Both internal and external evaluators: • Joint/participatory • Last as best

15. How to evaluate: Techniques • Participatory • Secondary data: • Project document, baseline, various reports • Primary data • Field visit; Interviews; FGDs; Observations • Photographic: Sounds, voices and pictures from the field: Heifer DVD example

16. The End Thank