Developing, Testing and Measuring Theories of Change: The WEIMI Experience to Date

1. Developing, Testing and Measuring Theories of Change:The WEIMI Experience to Date  WOMEN’S EMPOWERMENT IMPACT MEASUREMENT INITIATIVE Mary Picard, Consultant CARE USA, Gender Unit 31 Jan. 2012

2. Content • Who and what is WEIMI? • Progress on developing theories of change for a women’s empowerment program • Progress on testing theories of change • Progress on measuring theories of change • Spotlighting • Common challenges • Common issues • Next phase

3. Who and what is WEIMI? Core objective: to develop the necessary capacity, tools, guidance and practice to measure and demonstrate the impact of CARE’s work on women’s empowerment First developed by the Program Impact Unit; moved to the Gender Unit

4. Who and what is WEIMI? • During this period, 4 COs – Niger, Bangladesh, Egypt and Tanzania – were directly assisted through CO visits and considerable work done on their theories of change and indicators • Each of these countries has peripheral learning partners

5. Progress: Developing a Theory of Change • Measurability depends on the completion of these elements: Impact Goal Impact Group Sub-impact groups Domains of Change Pathways of Change Impact + DOC-level indicators Critical Hypotheses Breakthroughs Contextual Analysis A Coherent Set of Initiatives

6. CARE Tanzania Example • IMPACT GROUP: Marginalized and vulnerable women and girls living in rural underserved and environmentally restricted areas at critical life stages • Sub-impact groups: • Who are agriculture and forest-dependent • Mining or industry-dependent or living adjacent to mines • Fisheries-dependent • Pastoralist and agro-pastoralist

7. CARE TANZANIA THEORY OF CHANGE

9. Progress: Developing a Theory of Change • Other elements of a program are not conditional to testing the theory of change but complete the development of the program: • A Situational analysis • Identification of target groups • Identification of stakeholder groups • Strategies or promising practices

10. Progress: Developing a Theory of Change • Where are pilot countries now in developing the theory of change? • Still transitioning towards testing the ToC

11. Progress: Testing the Theory of Change • All countries have undergone some sort of matrix exercise to link the current set of initiatives to the theory of change, usually to pathways. It is one level of hypothesis testing.

12. Progress: Measuring the Theory of Change • Establishing an impact measurement system includes attention to: • Data Management • A knowledge management & learning strategy • Harmonization of M&E systems between projects and programs • A set of guidelines on M&E to align projects to programs • A central database that is rationalized to the program • Ongoing Analysis, Reflection and Learning • Staff Capacity and Systems

13. Progress: Measuring the Theory of Change • Progress on Data Management • All 4 COs have had an Impact Measurement Readiness Assessment (IMRA) done by Tom Barton • Increasing overlap and synergy between projects contributing to the same program • Niger has developed a KM & L strategy at CO level and progressed much further in data mgt • Some headway on IT front to facilitate shared access to documents

14. Progress: Measuring the Theory of Change Ongoing Analysis, Reflection & Learning • Which involves (a) structuring and systematizing the process of gathering people for reflection “in action” and “on action”; and (b) a process for routine documentation of learnings • The IMRA is a foundation for making needed changes. • COs are aware of what needs to change and are in the process of adapting their behaviors or routines.

15. Progress: Measuring the Theory of Change Staff Capacity and Systems • Almost all have re-aligned their staffing structure • Most are still working hard to re-align program support systems to the program approach • Most are aligning their design or new business with program theories of change

16. Spotlighting • Good exchange between pilot country and learning partners, e.g., Niger with Benin and Mali • Niger measured “self-efficacy” as an impact indicator in its baseline study for its CO ToC • Bangladesh will be testing a methodology for contextual analysis

17. Spotlighting • Tanzania finalized critical hypothesis: • Women in VSL groups build the self-esteem, social and economic capital that enables/shifts in gender roles in HH and community, which makes them to [sic] raise their voices and participate in the community decision-making. Both Niger and Egypt have a similar hypothesis to this

18. Common Issues • Domains of change that appear commonly: • Increased access to basic services and skills • Access to economic opportunities • Change in socio-cultural norms or practices • Civil society and/or government institutions are responsive • Common indicators: • % of men and women reporting ability of women to effectively control productive assets • % of men and women with changed attitudes towards gender-based violence • % of men and women reporting meaningful participation of women in public sphere (in adapted forms)

19. Common Themes • These appear explicitly at different levels of the theories of change: • Importance of laws that support gender equity • Women’s access to land (and other natural resources) • Gender relations in the household • Women’s access to SRH rights • The role of civil society actors in the social change process • Women and girls’ access to education services • Women’s participation in the public sphere

20. Common Challenges • Not enough time to focus on this • Getting focus – e.g., prioritizing and validating one or two critical hypotheses for testing • Finalizing and validating indicators • Understanding breakthroughs • Rationalizing pathways of change • Refining and prioritizing “critical hypotheses” • Facilitating a shared understanding of the work across the Country Office • Funding for this work

21. Next Steps • Include and capture the experience from learning partners • Finding points of connection with other initiatives in CARE (e.g., Pathways, WE-RISE) in order to consolidate experiences • Generating exchange based on common interest, within and beyond WEIMI COs / partners • Documenting the experience • Finding creative funding opportunities

22. THE END THANK YOU!