Frame work for Monitoring & Evaluation under IWMP

1. Workshop for Effective Implementation of IWMP Frame work for Monitoring & Evaluation under IWMP DOLR Govt. of India 7th &8th January 2014

2. Process Monitoring Input-Output Monitoring M&E Impact Assessment Components to be Monitored under M&E: Concurrent Process Monitoring Input – Output Monitoring Impact Evaluation Thematic/Special Studies Sustainability Assessment Case Studies/ Success Stories Feedback and Dissemination Mechanism for Learning Learning’s and Adoptability Envisaged Deliverables/Outputs Learning & Documentation M&E is an integral part of the project’s day to day operations rather than a periodic offline activity

3. 1. Concurrent Process Monitoring: Major Processes to be Monitored under IWMP • Entry point activity (EPA) • Sensitization and awareness • CBO formation & functioning • Capacity building • Transparency • Action plan preparation (Net plan & Complete action plan) • Action plan implementation (Activity implementation) • Flow of funds and procurement procedures • Income Generating Activities (Livestock / Skill based & Micro enterprises activities) • Institutional functioning- NGO’s, CBO’s & Government dept. or any other agencies. • Demonstrations if any • Withdrawal strategy • Convergence (with other projects) • Post project O&M strategy • Sustainability Separate indicators (Both qualitative and quantitative) to be developed for each of above processes

4. Project Stages and Process Monitoring Process Monitoring Cont…. A • Indicators • Prioritize • Identify new • Continue or discontinue old • Make changes (if needed) • Operationalise IV Stage I Stage Consolidation Initiation B D O Implementation capacitation II Stage III Stage C SHG (Indicators) “Indicators to be redefined as per the stage of the project” Weightage to be shifted as per project time chart

5. Process Monitoring carried out in Five-Stages In An Effective M&E the five stages are carried sequentially

6. Process Monitoring Cont…. A Format for Process Monitoring (Observation) Report - IWMP

7. Eg: Monthly Report Extract Taluk – Bidar BIDAR DISTRICT Sub-Watershed – Bagdal IWMP – 1/2009-10NGO – Santeshwari

8. Process Monitoring Cont…. Process Monitoring Methods & Reports • Reports on Monthly/ Quarterly basis • Reflect the actual ongoing field status. • All the process and activities will be monitored at random basis (at least 30% of Project site in a quarter) • Watershed wise reports will be consolidated at the district level • Report would also identify the key issues requiring immediate attention • Flag-off issues to be taken up as special studies or thematic studies. • Highlight the particular processes observed, the situational status, the reasons for the same and the institutions or individual associated in it. • Also suggest field based remedial measures to improve upon.

9. 2.INPUT-OUTPUT MONITORING

10. Under the concurrent input-output monitoring, the financial and physical progress of the project would be monitored. • Aspects such as fund release from the PIA to grass root level, actual condition/status of physical achievements, awareness creation, CBO’s capacity building, etc will be monitored using specific indicators. • This would be done through the MIS deployed and operationalised by the project. • M&Eunit would verify the MIS data integrity through field verification on sample basis through process monitoring. • Project will ensure flow of MIS data to M&E for analysis. • The MIS data thus received would be analyzed and the report would be made available to the top management once in six months.

12. 3. Impact Evaluation Address the overall impact on Natural Resources & Socio -Economic aspects Household level Short term Impact Community / Village Long term Micro / Sub Watershed APPROACH • Before (Baseline) • During (Midterm) • End of the Project • Post Project Before & After With & Without Continues….

13. Impact Evaluation Cont…. Continues….

14. Sampling Approach Impact Evaluation Cont…. Micro-Level (Micro watershed) Sub watershed State Village-Level (Household Survey)

15. Impact Evaluation Cont…. Baseline Study: • At the outset, as a part of impact assessment, Baseline Survey would be conducted in the sampled watersheds. • This analysis will provide a bench mark enabling a comparative analysis at the end of the project. • The degree of project goals and objectives achieved would be assessed as against this bench mark. • For a better comparison, ‘control samples’ outside the watershed (i.e. untreated area with the similar topographic and socio-economic condition) also be analyzed. Continues….

16. Impact Evaluation Cont…. End Project Impact Study: • At the End of the Project, M&Ewill also carry out the impact assessment to establish the net impact of the project on the socio-economic and natural resource indicators. • Data would be collected with respect to the same samples collected during the baseline. • A comprehensive comparative analysis to the baseline status would be made in this report. • The major perceivable impacts would be:- • Hydrological – Ground & Surface water, Drinking water status, etc. • Soil Status – Runoff, infiltration and moisture content, nutrient capacity, etc. • Agriculture and allied – Cropping pattern, cropping intensity, land reclamation, productivity and diversity • Institutional Aspects – CBOs, PRIs, any other • Socio-economic– Income level, Employment status, Migration, Women Empowerment, Educational status, Living condition, Livelihood options, etc. • Environmental– Land use land cover change, a forestation, biomass and carbon sequestration and • Spillovers / Externalities (if any) – Political involvement, linkages with other projects, financial linkages, etc.

17. Impact Evaluation Cont…. Sampling Strategy for Impact Assessment • Sampling will be Based on Ridge, Middle, Valley Concept • 25% of the projects will be studied • Households will be selected using PPS Method • Detailed questionnaire schedule covering relevant indicators will be used for Baseline, Midterm and End of the project enabling comparative analysis • Data will be collected by the M&E field staff • Satellite imageries will be used for spatial analysis

18. Impact Evaluation Cont…. Indicators for Impact Assessment Natural Resources Socio Economic • Surface Runoff • Water Resource Development • Ground Water level/Yield • Change in Irrigated Area • Crop Diversity • Crop Yield • Crop intensity • Fodder Availability • Afforestation • Climate Change & Biodiversity • Land Use Change • Sustainability of Structures /CBOs/IGA • Micro Enterprises • Employment Opportunity • Migration Status • Economic potential of Household Income • BPL Family • Animal Husbandry • Impact on Milk yield

19. 4. Thematic/Special Studies: • M&E will also conduct certain thematic studies on key issues as identified. • Some of the thematic areas include Capacity building, Investment pattern; CBO participation, women empowerment, gender issue, level of participation, Income generating activities; Run-off and land cover transformation, success stories, etc. • The basic principle governing the thematic studies are that • it should answer specific questions in the short run and help in policy derivations for next phase. • should reflect both positive and negative aspects of the project. • should be demand driven and locally identified (local specific) by the PIA. It is proposed that about one or two of thematic studies to be conducted per year. Continues….

20. Thematic/Special Studies cont….. A few possible Thematic / Special Studies • Investment Pattern • Social Inclusion • Women Empowerment • Capacity Building • Equity Issues • Common Property Resources • Income Generating Activities • Live stock • Short term Impacts • Sustainability • Run off Estimation • Land use / Land cover changes • Ground Water recharge • Soil Fertility • Water Quality

21. Thematic/Special Studies cont…. A Few New Thematic Areas (Suggestive) Community Based Activities Soil & Water Conservation Activities • Poverty Mapping • Sustainability Micro Enterprises • Input & Output estimate for a given Micro watershed • Soil erosion and Run-off estimation • Crop – Water Budgeting Hydrology Natural Resources • Groundwater level and quality • Reclamation of degraded land • Agriculture Productivity • Biomass Estimation • CDM supportive studies (Assessment of Carbon Credits) • Water Harvesting Structures: Potential of Water availability and its utility. Environmental Impact Assessment and Climatic Change Studies • Watershed impact on overall ecosystem and climate

22. 5. Sustainability Assessment: • Post project sustainability of assets generated and also the institutions created is very essential. • The level of involvement various institutions particularly the CBOs are crucial for the sustainability. • In the long run the maintenance of the assets has to be taken up by the community on their own. • Particularly the Common Property Resources (CPRs) needs to be maintained by the community. • It requires a support system built in by the project during the project period. • The withdrawal strategy needs to be evolved on these lines. • The M&E studies need to focus on the successful and effective functioning institutional arrangements for replication at a larger scale over time. • The withdrawal strategy adopted in the IWMP-I will be assessed for evolving a strategy in the next phase of the project. • Selected 10 per cent of the institutions from SHGs, CIGs, and AGs will be studied assessing their sustainability after completion of IWMP –I projects.

23. 6. Case Studies/ Success Stories: • Case studies to identify the specific project impacts would be carried out from time to time. • These case studies will reflect the immediate outcomes and outputs of the project with respect to specific area or sector. • The case studies reflect both positive and negative aspects. • This will help to consolidate on the success stories of the project and to improve upon the weak areas if any. • At least one case study will be prepared per project per year. 7.Feedback and Dissemination Mechanism for Learning: • Along with respective State departments identified M&E agencies evolve suitable dissemination mechanism for learning at different levels. • It will assist department in reviewing the M&E observations /findings, evolve compliance mechanism and provide feedback for decision-making. • Dissemination of the findings through learning events is proposed to ensure that available and accumulated knowledge is fed back into an operational learning process that provides appropriate and timely information to relevant stakeholders, including management, to allow informed participation and on-going corrections/adjustments.

24. 8. Learning’s and Adoptability: • The regular feedback of Process Monitoring Reports will provide online corrections in the course of project implementation itself. • The regular dissemination of information both from Process Monitoring and Impact Assessment if any would enhance the corrective options. • The thematic studies carried out at a larger scale discretely would support the policy corrective measures to be adopted as a future strategy. • Annual learning workshop would be organized at each district level and at the project level (State Level) to exchange the M&E findings arrived at through various monitoring and assessment mechanisms. • The End of Project reports would lead to a State level Workshop disseminating the major findings which would guide in developing new policy requirements if any. • Based on the experience of various States a National level workshop (Annual/Once in two years) can be planned.

25. 9. Envisaged Deliverables/Outputs: • The following outputs / reports will be envisaged during the project period, from the M&E Sector. • Inception report • Process monitoring reports (Monthly/Quarterly, Half yearly/ Annually) • Baseline & Impact report (for the sampled watersheds) - Preparatory phase Evaluation report, Work phase Evaluation report and Final Evaluation report. • Input – Output reports (Annually) • Thematic reports (Discrete) • Case studies/Success stories

26. Monitoring & Evaluation (In Nutshell ) Discrete monitoring Concurrent monitoring Impact Assessment Self Evaluation/ Assessment Process Monitoring Input Output Monitoring Participatory Monitoring Independent M&L By CBOs (SWS-EC, AG & SHG) facilitated by NGO (Frequency) Community and household level Various levels through MIS Data Continuous Monitoring Yearly Quarterly Pre treatment Baseline Mid –term impact In addition Thematic Studies, Case Studies and other related studies (Special) if any will be prepared as directed by VWDA End of project Post Project

27. Conclusion: • M&E is a part of the project not an outsider • M&E is a facilitator of learning and is not for policing • M&E is an effective management tool • M&E ensures very high level of transparency MEL &D is the eyes and ears of the project

28. How an M & E can be successful? • Good Integration & Co operation between M&E & Project Management • M&E Information should be unbiased, relevant, timely & accurate • M&E documentation should address or resolve problems • M&E staff should have good skills and experience • Monitoring and Evaluation should be integrated – Single agency • Project Evaluations and Assessments – at frequent intervals • Information Dissemination – Upward & Downward • M & E also plays a facilitator’s role for all stakeholders • M&E should be an integral part of the project “M&E is not - & cannot be - a substitute for good management. For M&E to be success – it needs to be driven by manager’s need for information, their use of information and their desire to create a learning environment”

29. Road ahead • Put in place an independent M&E agency—Feb-2014 (ToR and Bid documents uploaded on DoLR Web site) • Ensure the critical elements of Concurrent –Process monitoring in real time • Arrange M&E workshop at SLNA level (April 2014 • Realise importance of learning and policy linkages • DoLR to contract directly Regional Independent M&E Agencies Note the Change in the receptive level and Body language of DoLR- “Reciprocate”

30. Organizational Structure Proposed MEL&D Structure HEAD QUARTER *Project Director (M&E) • Natural Resource Scientist • Hydrologist • Social Scientist • Statistics Scientist • Remote sensing/GIS/IT Scientist • Documentation Specialist AT PROJECT LEVEL District Coordinator (1 for 3 districts) Natural Sciences / Social Sciences Field Staff (2 for each District) Social Sciences Natural Sciences * A senior person among the scientists will be act as Project Director

31. ANNEXURE –III Multi-faceted Indicators of Impact at Different Spatial Scales (In Case of Soil Conserving Technologies)

32. ANNEXURE – IV A Format for Process Monitoring (Observation) Report - IWMP

33. ANNEXURE – V EXPECTED / ESTIMATED OUTCOME

34. ANNEXURE - VI LOG MATRIX FOR KEY INDICATORS TO BE ASSESSED UNDER IWMP This is only an indicative format for operationalization of project assessment through ME&L activities. However, this is not a final document; it can be used as guide to conduct various activities under ME&L. An improvement on this matrix can be made as per the actual field requirements. (For Neeranchal project additional indicators will be added and details would be worked out in the format after inputs from various stakeholders)

35. When You Have Measurement, You Have PERFORMANCE When You Have NO Measurement, You Have EXCUSES What Gets Measured, Gets Managed Well

36. Thank You