Nepal - Assessing the Social Impact of Rural Energy Services Alternative Energy Promotion Center

1. Insert relevant PwC graphic in this space Nepal - Assessing the Social Impact of Rural Energy Services Alternative Energy Promotion Center Monitoring and Evaluation Framework November 2009 *connectedthinking Pwc

2. Table of Contents Section Page Development of M&E framework – the context 1 5 Development of M&E framework – our approach 2 10 Familiarisation and sensitisation 3 12 Review of existing M&E framework 4 15 Proposed M&E framework 5 20 Process flow for MIS Reporting and Evaluation 6 35

3. AGENDA of the programme Day 1: Nov 5th 9:15-9:30 - Welcome and Introduction of participants  9:30:9:45 - Introduction and Objective of the Workshop, Dr. N.P. Chaulagain  9:45-9:50 - Inauguration of the Workshop, Honorable Minister, Thakur Sharma  9:50-10:20 - Overview of M&E framework for AEPC, RakeshJha, PwC 10:10-10:50 -Remarks: By all program officials 10:50-11:00 - Vote of Thanks and Closing of the session, Dr. N.P. Chaulagain  11:00-11:15 - Tea Break Technical Session-I  11:15-13:00 - Introduction to Management Information Reporting & Monitoring System (MIRMS) Mr. Rakesh Jha and Mr. Manoj Mania, PwC  13:00-14:00 - Lunch Break  14:00-15:00 - MIRMS: Manual Data Entry and upload provisions for all renewable technologies, Mr Manoj Mania, PwC 15:00-15:45 - MIRMS: MIS reporting, Mr Manoj Mania, PwC 151:450-16:00 - Tea/ Coffee break 16:00-17:30 - Project Monitoring – MH programmes of AEPC, Mr Manoj Mania, PwC

4. AGENDA of the programme Day 2: Nov 6 Technical Session-II  9:00-10:00 - Demonstration of MIRMS: Scorecards, KPI Monitoring & Dashboards – Mr. Rakesh Jha and Mr. Manoj Mania, PwC 10:00-11:00 - Training and live run, Mr Manoj Mania, PwC 11:00-11:15 - Tea/ Coffee break 11:15-12:45 - Training and live run contd…Mr. Manoj Mania, PwC 12:00 – 13:00 - Concluding remarks and vote of thanks, AEPC 13:00-14:00 - Lunch 14:15 - Departure to AEPC

5. Section 1 Development of M&E framework – the context

6. Context Electrification target under the Water national Plan , 2005 • Low electricity access in Nepal – 35% total; Urban- 85%; Rural – 30% • Low per capita electricity consumption – 76 unit • Cost of grid connection not feasible at all places – Renewable can play a major role • Grid extension in majority of villages not viable - poor financial condition of NEA Source: NEA Annual Report, FY 2008 -09

7. Context • Nepal endowed with huge potential in hydropower (>40,000 MW) and other Renewable energy sources. • Increasing investment in hydropower – GoN, donors • AEPC created in 1996 to promote renewable in the country.

8. Context • Diverse programmes – Anecdotal evidence of positive impact but little systematic quantification of results on the ground • WB funded study on ‘Assessing the Social Impact of Rural Energy Services’ – Oct 2008 -Sep, 2009 • Systematic M&E framework to assess the development impact of MH schemes • Update the business plan for AEPC

9. Why M&E Improving internal management • Better planning and budgeting • Better implementation and reporting Generating support • Respond to needs of stakeholders • Explicit commitment to track contribution to global indicators e.g. MDGs Ensuring accountability • Demonstrating results against money spent - Demanded by govt., donors, NGOs • Ultimately results on the ground matter - not implementation progress Just in case: How M&E helps ME? • Motivates by seeing the difference it makes for people • Learn from experience: Build on success and learn from failures… • Incentives for better performance…?

10. Section 2 Development of M&E framework – our approach

11. Section 1 – development of M&E framework Development of M&E and MIS system – our approach

12. Section 3 Familiarisation and sensitisation

13. Review of policies, regulations and impact studies Interaction with REDP, ESAP and AEPC officials Field Visit Dhunne Khola (REDP) Tanahun and Banglung (ESAP) Complete overview of existing M&E Section 2 – Familiarization and Sensitization Familiarization of the context and the program by the project team Focus Group Discussion at Dhunne Khola Micro Hydro site

14. Section 2 – Familiarization and Sensitization Project sensitization of the AEPC staff Organized a sensitization workshop at the outset of the project DAY 1: November 18, 2008 • Introduction and Context • Overview of M&E Concepts • Developing the Monitoring Framework for AEPC • The Role of Evaluation DAY 2: November 19, 2008 • M&E framework for Assessing the Social Impacts of Rural Energy Services • Monitoring: Key Performance Indicators • Evaluation: Survey Conceptualization and Design Sensitization workshop at AEPC

15. Section 4 Review of existing M&E framework

16. Section 3 – Review of existing M&E framework M&E diagnostic review - REDP

17. Section 3 – Review of existing M&E framework M&E diagnostic review - ESAP • Quarterly reports are sent to AEPC with detail work plan based on the results of the survey. • Each installation is surveyed after a month of installation by the LPO and the promoter. • Technical evaluation – after 3 to 4 months by the LPO and RRESC staff. • Mid term review of the annual work plan to check whether program objectives are met. • Evaluation of LPOs by RRESC • The field technical coordinator from the RRESC office submits progress reports to AEPC by the 12th of every month – based on pre-defined formats

18. MIS – Existing & Proposed for REDP Existing Focus more on data gathering Targets not reflected in MIS Manual calculation of formula fields in few formats Project monitoring report captured only status of completion of activity Proposed MIS aligned for performance monitoring Captures targets for each measurable indicator and helps evaluation System calculates formula based on dependent fields System would capture date of completion to highlight delay in implementation, if any Section 5 – Development of MIRMS

19. MIS – Existing & Proposed for ESAP Existing Client server mode visible to select group of people within AEPC office(max 8 to 10 users) – local installations required Absence of integrated information system - separate databases Targets not reflected in MIS KPIs not linked to performance management of business units Requires design change in system for meeting any future requirements Static reports with no feature to compare and perform trend analysis Proposed Web based system – information available to all users at any location having internet connection Integrated AEPC database -warehousing periodic information for analysis and evaluation in the system itself Target driven system for measuring performance every month Dynamic system to incorporate any future requirements Multiple ways of viewing data and conducting analysis Section 5 – Development of MIRMS

20. Section 5 Proposed M&E framework

21. Primary reporting interface for meeting all the reporting requirements of the management(financial, technical, operational, projects, community participation, capacity building) Eliminate the burden of disparate electronic and paper based data collection requirements Provide the top management specific information about the programs and its business units at a glance. Enabling the management to drive strategy by evaluating performance against the KPIs. Protect the confidentiality , availability and integrity of data Section 5 – Development of MIRMS MIRMS - what it offers

22. Section 5 – Development of MIRMS MIRMS – leveraging existing capability of AEPC • Using existing computer infrastructure of AEPC server for hosting the system on the internet • Utilising existing manpower and computer infrastructure at the field level • Supplements existing information base of AEPC to enhance decision making abilities • Integration with REDP and ESAP databases for information consolidation • Existing formats have been retained and new formats designed and aligned to the KPIs in the proposed M&E framework • Reduction of work load on M&E staff by leveraging the information for performing evaluation and meeting top managements reporting requirements

23. Section 5 – Development of MIRMS MIS - Design and Reporting • Comprehensive MIS formats designed to facilitate regular review of project activities for all RETs • Structured and unified formats for ease in data gathering • Mapping of data fields to existing transactional system • Yearly / Monthly Targets to be set in the system to compare progress towards achieving outputs • Flexibility in reporting and analyzing the data • Trend analysis • Comparative analysis • Dynamic reporting system with no static reports – user can extract reports in desired format at any given point of time from any location

24. MIRMS will interface with existing systems either through on-line or off-line mode and extract information User friendly screens provided for mapping and uploading of external data source to MIRMS Information extracted either on monthly or trimesterly basis depending on availability of information Section 5 – Development of MIRMS MIRMS – Mapping with existing database of AEPC

25. Build on the existing infrastructure - optimal utilization Existing AEPC server SQL server database Developed on a 3-tier web technology framework comprising of java technology integrated with MS SQL server Leveraged industry best practices for designing and developing a scalable system Creation of a centralized database repository MIS formats designed for all programs Section 5 – Development of MIRMS Development of MIRMS

26. Section 5 – Development of MIRMS Proposed MIRMS

27. The proposed M&E framework is aimed to: Align and design to meet AEPC’s vision, mission and strategic priorities. Use M&E as a strategic planning tool. Optimise the existing set up Establish a drill down feature for analysis of information Generate MIS reports at every level Top Management, Programme Managers Department head and District Level Monitoring Field based operation Section 5 – Proposed M&E framework Institutional arrangement for M&E

28. Project Sites Middle Management Top Management Level of Management Section 5 – Proposed M&E framework Programme Performance Indicators/ Strategic Objective Parameters Functions: Operational/Technical/ Financial/Capacity Building Transactional Activities/ Complaint Status Process Performance Indicators Need based MIRMS Advantages of MIRMS Review Timeline Monthly/ Trimesterly Monthly/ Trimesterly/Yearly Monthly Functions: Operational/Technical/ Financial/Capacity Building • Monthly monitoring of operational data is done by the field staff responsible for the respective functions. • Trimesterly monitoring of performance parameters - by the district heads or monitoring officers of respective programs • The top management reviews the program and strategic parameters on a monthly, quarterly or yearly basis to take decisions and decide the policy level interventions.

29. To evaluate the performance of the project against defined objective/target Facilitate review and comparison across project/ location. Useful for setting performance targets and efficient assessment. Input indicator mostly the programme fund support, contribution in kind and technical assistance by community and awareness generation programmes Output Indicators mostly the number of projects/systems installed, capacity enhancement of the target beneficiaries and the community mobilization process. show the benefits accruing to the community through the awareness generation and mobilization process. Outcome Indicators indicators that mostly are a measure of the affordability, operational efficiency and reliability of the MH programs. Section 5 – Proposed M&E framework Development of KPIs

30. Section 5 – Proposed M&E framework KPIs – Top Management (MH – REDP & MGSP) • No. of MH projects commissioned • Installed capacity of MH • Percentage coverage of target population through awareness generation programme • Progress against projects schedule* • Project implementation delay* • Utilization of Funds • Percentage of HHs with electricity access through MH • No. of enterprises with access to MH • Value of proposed equity by the community (equivalent to labour and cash amount) vis-a-vis actual contribution • % of community facilities with access to other RETs • % of community facilities with access to MH • Percentage of Complaints resolved (consumer level) in stipulated time period • Power availability* • Collection efficiency*

31. Section 5 – Proposed M&E framework KPIs – Top Management (Biomass) • Achievement of ICS installations • Achievement of Metallic Stoves installations • Achievement of Gasifier installations • Percentage coverage of target population through awareness generation programs • Total subsidy allocation • Utilization of Funds • % Achievement of Training programmes for program staff • % Systems functioning during verification • % of community facilities with access to MH • System performance in ICS • System performance in Metallic Stoves • System performance in Gasifiers

32. Section 5 – Proposed M&E framework KPIs – Top Management (Solar) • Achievement of Solar systems installed • % subsidy disbursement • Avg. delay in subsidy disbursement in all Solar systems • Percentage coverage of target population through awareness generation programs • System performance for ISPS, SHS & PVPS • % of complaints resolved in stipulated time period

33. Section 5 – Proposed M&E framework Proposed Institutional arrangement for M&E

34. Section 5 – Proposed M&E framework Snapshot of MIRMS

35. Section 6 Process flow for MIS Reporting and Evaluation

36. Periodic Data capture through MIS formats Monthly MIS Reporting MH Project Monitoring KPI Analysis & Scorecards for Business Units Performance Review through Dashboards Section 6 – Process flow for MIS Reporting and Evaluation

37. Section 6 – Process flow for MIS Reporting and Evaluation Monitoring & Evaluation Framework

38. Section 6 – Process flow for MIS Reporting and Evaluation Data Capture • Populate monthly data in MIRMS through two options: • File uploads – mapped to existing AEPC databases • Manual Entry – data entry screens • Periodicity of data capture • One time – DPR, Funding programs, etc • Monthly • Trimesterly • Annually • Data captured at different levels of the organization depending on data availability

39. Section 6 – Process flow for MIS Reporting and Evaluation MIS Reporting • Dynamic reporting engine supporting multiple ways of viewing data and conducting analysis

40. Section 6 – Process flow for MIS Reporting and Evaluation MIS Reporting • Perform trend analysis on any selected parameter

41. Section 6 – Process flow for MIS Reporting and Evaluation MIS Reporting • Consolidated view of past performance data

42. Section 6 – Process flow for MIS Reporting and Evaluation Project Monitoring • Tracks project schedules and assists program managers to take corrective measures

43. Section 6 – Process flow for MIS Reporting and Evaluation KPI Analysis • Single window for top management to evaluate performance through KPIs

44. Section 6 – Process flow for MIS Reporting and Evaluation Program Dashboards • Single view of performance data in graphical form to different levels of user

45. Section 6 – Process flow for MIS Reporting and Evaluation Sample - Dashboard

46. Section 6 – Process flow for MIS Reporting and Evaluation Snapshot of dashboards • The dashboards would facilitate the Program Mangers and M&E staff to monitor project development • designed to provide specific information about the performance of each component of the programme and present critical measurable parameters of each program at a glance. • The dashboards would provide a graphical display of the overall performance of the programs on key performance parameters like achievement of programme objectives, community reach, financial performance, etc. • The dashboards can be generated starting from the zonal level to individual level dashboards for each project/[programme

47. Section 6 – Process flow for MIS Reporting and Evaluation How to migrate to MIRMS • Form a core group of people for taking ownership of MIRMS and formulating strategies for information • Identify at least one district form each region for testing and roll out • Run this parallel to the existing M&E arrangement • Develop a team of ToT • Handholding of people engaged in M&E by ToTs • Use of operation manual • Periodic review of the system – frequency of monitoring on the higher side in the beginning • Assess performance of the M&E team and MIRMS • Take appropriate measures as per requirement – training, modification in MIRMS

48. pwc

49. Regulatory and policy overview