Impact Evaluation for Human Rights Interventions

1. Impact Evaluation for Human Rights Interventions

2. Development of impact evaluation methodology for projects aiming to reduce gender based violence Main considerations: • Need for rigorous impact evaluations • Human rights interventions / agendas / philosophical priorities (process-focused, etc.) • Constraints of community-based projects (CDD) Mission impossible?

3. Human Rights Based Approaches • Human Rights Initiatives Initiatives having the improvement of human rights as their primary goal • Rights Based Approaches Approaches that focus on the awareness of rights and empowerment as a means to achieve development goals

4. Main Differences • Human Rights Initiatives: VAW Outcome: the improvement of human rights Means: individual and collective empowerment plus institutional protection • Rights Based Approaches: GBV Outcome: improved participation in, and benefits from, development processes Means: individual and collective empowerment through awareness of rights

5. Change model Inputs / Outputs Intermediate Outcomes Intermediate Impact Ultimate Impact Change in awareness, knowledge & practice ______ Intervention level: Social services education / health tribal leadership labor ----------- Women’s grassroots groups, networks & professional associations ----------- Government ministries judicial system law enforcement Inputs: Action research information / communication / capacity building networking / mobilization __________ Outputs: # of research products / workshops / seminars / media events / networks / advocacy groups Effective implemen-tation and use of policies and laws protecting women from violence Reduced incidence of VAW -------- Improved participa-tion in and access to develop-ment processes

6. Intervention strategies • Awareness raising • Advocacy • Capacity building • Legal literacy • Gender sensitivity • Violence prevention • Action-oriented research

7. Target population Intermediate impact • Local organizations & advocacy groups • Social service agencies • Judicial system • Security forces, police • Research groups Ultimate impact • Women survivors / potential victims

8. Evaluation Experience • UNIFEM’s definition of impact evaluation for grantees:A project impact assessment evaluates to what extent your expected results were achieved and what these results mean to those involved in and/or affected by the project. • UNIFEM’s Trust Fund allocates 10% of each grant to impact evaluation. • Grantee interpretations of impact evaluation are very broad, are output and process oriented, and are usually based on qualitative self assessments. • Most projects lack sufficient precision in defining objectives, target populations and indicators.

9. What is impact evaluation? The guidelines differentiate between the more rigorous classical form of impact evaluation that refers to the measurement of net effects and proof of causality, and outcome or summative evaluation that refers more generally to the end results of an intervention and which relaxesthe proof of causality requirement.

10. What is rigorous impact evaluation? Within the larger family of summative evaluations, impact evaluation is specifically charged with delivery of proof of causality in order to become a defensible basis for modified policies, program extensions and replication. At its most rigorous, it follows the dictates of experimental design in social science research and can become considerably demanding and costly.[2] To date, for instance, a relatively small proportion of World Bank projects are evaluated in accordance with such standards.[3] [2] The recently estimated cost of rigorous impact evaluations for World Bank projects ranges from $200,000 to $900,000 each (OED and Impact Evaluation – A Discussion Note, 2005). [3] The MIT Poverty Action Lab notes in 2004 that only 2% of all World Bank evaluations conform to the rigors of randomized control design (reported in the New York Times, July 2004).

11. Constraints • Funding (average grant $ 50-75,000) • Time(average duration 1.5 - 2 years) • Capacity/motivation(lack of knowledge/ mistrust of conventional evaluation approaches) • Data(absence of baseline, secondary, monitoring data) • Complexity (cultural, contextual, dimensional)

12. Evaluation design options • Randomized design (RCTs) • Quasi-experimental design / matched control group design • Ex-post comparison of project beneficiaries with control group • Rapid assessment or self evaluation, conducted ex post

13. Criteria to determine candidacy for impact evaluation • Overall strategic importance of project • Policy implications • Stakeholder interest • Scaling up potential • Evaluability of the project • Quality of project design • Capacity and motivation of grantee

14. Recommendations • Develop strategy and plan for rigorous impact evaluation in advance (Part I: Decision Guidelines) • Strengthen project design capacity including the development of indicators (Part II: Methodological Guidelines) • Use mixed-methods approach (Part II: Methodological Guidelines) • Strengthen monitoring capacity (Part II) • Publish evaluation results