Cornwallis XVI Analysis for Future Conflict

1. Cornwallis XVI Analysis for Future Conflict Innovative Approach to Evaluating Interventions in Fragile and Conflict-Affected States: The Helmand Case Study

2. Contents • Introduction • Challenges to evaluation in the Helmand context • HMEP: a cutting-edge M&E solution • Conclusions: Future prospects for M&E in fragile states

3. 1 - Introduction Economic context • Sharpest spending cuts since WW2 • Cuts worth around £128 billion over 4 years • 30-40% cuts for some Departments • DFID… Ring-fenced !  What does that mean for the future ? • Extra scrutiny to ensure DFID’s lucky escape is justified • Focus on Value for Money for every taxpayer £

4. 1 - Introduction Helmand Context • Extra pressure for results-based M&E • Pledges to withdraw forces associated with successful, visible stabilisation • Everyone is watching closely: General public, Media, MPs, Cabinet Office • Need to go beyond outputs to look at outcomes and impacts • Need to communicate information to multiple audiences quickly and effectively

5. 2 - Challenges to Evaluation in Helmand Context These fall into 3 main categories: • Political and Institutional • Logistical • Cultural

6. 2 - Challenges to Evaluation in Helmand Context Political and Institutional Challenges: • Weakness, non-existence or fractured nature of government systems • Institutional complexity of the operating environment in Helmand • The pre-eminence of political and military considerations • Targeted interventions not for universal benefit • Pressure to deliver ‘good news stories’

7. 2 - Challenges to Evaluation in Helmand Context Logistical Challenges: • Data is almost always patchy and of poor quality • Freedom of movement is significantly curtailed • Association with ISAF puts enumerators’ lives at risk • Negativity about the feasibility of conducting research in Helmand • Difficulties in establishing institutional memory

8. 2 - Challenges to Evaluation in Helmand Context Cultural Challenges: • Social desirability bias particularly high in Helmand • Difficulties associated with reaching women

9. 3 - HMEP: a cutting-edge M&E solution Project Background: • The Helmand Monitoring & Evaluation Programme commissioned by DFID and the Provincial Reconstruction Team (PRT) in late 2009 • 4 key outputs • Establish baselines • An operational, up-to-date, user-friendly database and GIS database • New knowledge and recommendations • Improved programming capacity

10. 3 - HMEP: a cutting-edge M&E solution Established Baselines • Discrete primary data collection required to support measurement of logical frameworks. • Attention was given to developing quantitative, SMART indicators • Data collection methodology was designed taking into account lessons learned

11. 3 - HMEP: a cutting-edge M&E solution Carefully Designed Sampling Strategy: • 2004 census data and the Central Statistics Organization’s (CSO) raw data on settlements posed several issues • District boundaries changed over time • Location of CSO settlements contained spatial coordinate errors • Population per settlement from the CSO data did not always reflect what could be seen on the ground through spatial imagery

12. 3 - HMEP: a cutting-edge M&E solution Carefully Designed Sampling Strategy: • The decision was taken to target males Heads of Households (HOH) • Number of 10 people per household was assumed • Profiling results from different waves of the same HOH survey will be compared with one another to establish HOH profiles • Multi-staged random probability sampling process was used with a random route for selecting households in each sampling point • HOH survey is implemented by an Afghan survey partner

13. 3 - HMEP: a cutting-edge M&E solution Robust Sample Sizes at Provincial and District Level :

14. 3 - HMEP: a cutting-edge M&E solution Longitudinal ApproachProviding Confidence in Time-Series Analysis: • Essential to ensure: • Consistent sampling point selection method • Continuity in the questionnaire rolled out each quarter • Building a panel proved impossible in Helmand

15. 3 - HMEP: a cutting-edge M&E solution A Combined Quantitative and Qualitative Research: • Qualitative research to complement quantitative findings • Capture views of women • Explore the most puzzling trends resulting from the quantitative research • Provide a more in-depth understanding of the ‘story’ and causal factors behind some of the less straightforward trends in perceptions

16. 3 - HMEP: a cutting-edge M&E solution Analytical thinking and theory of change

17. 3 - HMEP: a cutting-edge M&E solution

18. 3 - HMEP: a cutting-edge M&E solution Presentational Features

19. 3 - HMEP: a cutting-edge M&E solution Database & Website:

20. 3 - HMEP: a cutting-edge M&E solution GIS Mapping:

21. 3 - HMEP: a cutting-edge M&E solution Intelligent use of information: • HMEP supports more intelligent use of information in assessments, planning and daily operations: • look back on a quarterly and annual basis on progress towards strategic goals • look forward in terms of planning and future strategic direction • supports PRT’s daily operations through improved information management in the collation and centralisation of relevant data and its storage in a centralised and accessible information management system.

22. 4 - Conclusions: Future prospects for M&E in fragile states • HMEP’s specificity lies in its holistic approach to implementing an M&E system in a part of the world where it is crucially needed but difficult to achieve. • Before assessing the replicability of the HMEP model, it is essential to understand what made it possible in the first place. • Most important factor of success : Presence of a PRT • Important to recognise that each province, region and country contains its own, specific set of contextual issues and challenges • No magic formula, but it proves that meaningful and sustainable M&E systems is possible even in the most difficult environments

23. HelmandMonitoring and Evaluation Programme(HMEP) Christine Kolbe: christine_kolbe@coffey.com