1 / 25

Index

PUBLIC SERVICE M&E NETWORK RE-LAUNCH National Treasury 14 May 2007 Monitoring, Evaluation & Reporting – an essential tool for sustainable service delivery A case study on water sector Presented by: Kalinga Pelpola and Telly Chauke. Index. Background and context

marnin
Télécharger la présentation

Index

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. PUBLIC SERVICE M&E NETWORK RE-LAUNCHNational Treasury14 May 2007Monitoring, Evaluation & Reporting – an essential tool for sustainable service deliveryA case study on water sectorPresented by:Kalinga Pelpola and Telly Chauke

  2. Index • Background and context • Moving from project, programme to SWAP • Masibambane Programme – Vehicle for sustainable, SWAP to service delivery and M&E&R • Development of M&E&R to support sustainability of the Masibambane Programme • M&E&R Indicator and Framework Development • Water and Sanitation Spotcheck Assessments • Quality of Life Evaluations and External Evaluations • Challenges • Conclusion

  3. 1. Background and Context • Pre 1994 Water services • Fragmented management- homeland and self governing states • Department of Water Affairs and Forestry (DWAF) – only responsible for water resources • Ineffective or lack of LG especially in rural areas • Post 1994, • Review legislation leading to WS Act • addressing services backlog to marginalised communities; • Water backlog – 14million people • Sanitation backlog – 24million people

  4. 2. Challenges facing the Water Sector in South Africa (post 1994) Challenges post 1994 democratisation: • Address the services backlogs and social injustices of post apartheid South Africa; • Improve quality of life of previously marginalised communities; • Create institutional structures and delivery mechanisms for coordinated and collaborative approach to services delivery; • Provide an enabling environment for stakeholder participation and alignment of inter-department and inter-government involvement; • Establish frameworks for measuring and reporting service delivery implementation; and • Ensure sustainability of services provision.

  5. Moving project to programme to Sector wide approaches MULTI-SECTORAL Gearing-up Programme Man MSB II MSB I CWSS RDP Project Man 1994 1996 2001 2004 2007 Multi-Sectoral Focus SWAP Focus Project to Prog: Focus

  6. 3. Masibambane Programme – Vehicle for sustainable, sector wide approach (SWAP) to service delivery and M&E&R development • The overall objective of the Masibambane Programme was in line with the Millennium Development Goals to improve the quality of life and contribute to poverty eradication; • Masibambane (meaning “let’s work together”), was led by DWAF and supported by the donor organisations and further supported the continuous development of a water sector M&E&R strategic framework; • The M&E&R strategic framework is supported by the SWAP. • The Masibambane SWAP ensured the collaboration & participation of: • govt. departments – DWAF (sector leader), Dept. of Provincial and Local Government, National Treasury, Education, Health, Housing; • Civil Society Organisations (CSO’s); and • the South African Local Government Association (SALGA)

  7. MSB : SECTOR WIDE APPROACH SA Strategic Objectives MDGs Need for Accelerated delivery MASIBAMBANE SWAP Multi-Sectoral Approach

  8. MOVING FROM PROGRAMATIC TO SWAP • WHAT IS SWAP? • Is defined as a way of working together between government and development partners • The aim is : • to broaden Government ownership over public sector policy and resource allocation decisions within the sector, • to increase the coherence between policy, spending and results; and • to harmonise donor requirements into Govt processes thus reducing transaction costs. • It involves progressive development of a • comprehensive and coherent sector policy and strategy, • unified public expenditure framework for local and external resources; and • common management, planning and reporting framework.

  9. Government- Led process of Donor Coordination Agreed process for harmonization of systems SWAP Systematic Mechanism for Consultation of beneficiaries Clear & agreed Sector policy And strategy Common Performance Monitoring/ reporting Sector mtef (all local and External Resources) MOVING TO SWAP SWAPs typically have six components:

  10. MOVING TO SWAP (cont) 7 assessments for a SWAP • Macro-economic • framework 7. Institutions and capacities 2. Sector policy and national strategic framework SWAP 3. Medium term expenditure framework for the sector 6. Performance monitoring & client consultation systems • Donor • coordination • systems • Accountability & • public • finance management • systems

  11. Impact of MSB on expenditure? Gearing-up BoTT Impact 1 MILLION th MSB CWSS

  12. 4. Development of M&E to support sustainability of Masibambane Programme • Sustainability of MSB programme dependent on monitoring, evaluation and reporting of measurable sector indicators and targets: • physical services delivery; • financial execution; and • “soft-issue” components (such as gender, training, employment creation and environment) • M&E&R targets were defined in the Strategic Framework for Water Services: • national for strategy and policy; • regional for tactical planning; and • local Water Services Authorities for implementation and operations

  13. The water sector M&E&R strategic framework is embedded in the context below

  14. Monitoring Framework

  15. 5. M&E&R Indicator Development • Incremental development of M&E&R system • Initially main DWAF indicators focused on measuring services aimed at backlog eradication: • Number of people served with water supply to the standard set; • Number of People served with sanitation facilities; • Number of schools and clinics served with water and sanitation facilities; • Number of bucket toilet systems eradicated. • From 1994 to 1998 additional indicators were included to monitor amongst others: • employment creation; • gender equality; • Training; • health and hygiene promotion, etc • In 2003/2004, sector stakeholders developed a comprehensive list of indicators to monitor and evaluate the effectiveness, efficiency and sustainability of the programme aligned to the deliverables described in the Strategic Framework for Water Services.

  16. Incremental development of the Indicator reporting • Indicators were added to the report during each reporting period • The selection of indicators added was prioritised according to requirements of the EU and Sector Departments, as well as availability of data • Start of with quantitative information (Year 1) • Expand to qualitative information found commonly within Departments (Year 2) • Add water services specific indicators required (Year 3)

  17. Conceptual development of the quarterly reporting

  18. Breakdown of development per Indicator Framework Topic

  19. 6. Water Sector M&E&R Framework Development • Further support for the development of a sector M&E&R strategy was contained in a proposal by the Presidency to operationalise the GWM&ES, wherein it was stated that “…all public service entities need to undertake their own credible M&E processes that meet clearly defined standards and deliver information on their progress and performance.” • DWAF, as the water sector leader (on the success of the CWSS and other Departmental M&E&R programmes), embarked on the development of a robust water sector M&E&R strategy framework, to address the challenges of a transforming water sector, the requirements by donor agencies, and the priorities of the Presidency.

  20. 7. Water & Sanitation Monitoring (spotcheck assessment) • DWAF has undertaken independent external spot-check assessments on water and sanitation projects implemented by the sector piloting the projects executed through the dplg MIG programme.. • The spot-checks focused on the standards, norms and quality of water sector projects. • Rigorous Scientific methodology was employed in the research study, tools, project sample size. • The results and analysis of the spot-check assessments is aimed at guiding the sector in understanding the quality of delivery mechanisms. • The 2006/07 spot check project sampled assessed 10% of the total population of water & sanitation projects in the database utilized. Due to time and resource constraints, the initial exercise was conducted by the CSIR.

  21. 8. External Impact Assessments (Process & Quality of Life Evaluations) • For the purposes of DWAF’s M&E&R processes, quality of life means a personal statement of the positivism or negativity of multiple attributes that characterise one's life. It is personal. It is subjective. It may vary from person to person. It may change from day to day, but it is the best indication of overall satisfaction or assessment of services received or not received by the community members. • The QoL evaluations aim to investigate the impact of water and sanitation projects/programmes on beneficiary communities, as well as assess the efficiency, effectiveness and sustainability of such projects/programmes. This research is scheduled to commence in the 2007/2008 financial year. • The methodology for conducting QoL evaluations will involve: • Process evaluations, through one-to-one interviews and focus group sessions; • Desktop study and literature reviews; • Impact assessment on beneficiaries; and • Consolidation of analysis and reporting.

  22. 9. Reason behind external Evaluations • Internal progress monitoring and reporting is performed on a monthly, quarterly and annual basis by sector stakeholders and is subjective in nature; • External independent evaluations, can be conducted by local and or international experts. • the effectiveness, efficiency and impact of the programme against the indicators formulated by the sector. • The continued relevance and sustainability of the programme is also evaluated to determine whether the programme should continue or whether programme funds should be directed elsewhere; • and to make recommendations for improvement. • Best practice suggests a combination of both internal and external assessment of successful outcomes.

  23. 10. Challenges • KPIs used in the early years of the programme were effective in measuring whether targets were achieved, but did not cover quality or sustainability issues; • “double counting” of beneficiaries a real problem, specifically between dplg and Dept. of Housing, who each have its own M&E system; • Integration of KPI’s of DWAF and dplg (as well as the other sector players) has been problematic as each of the systems had a different focus. • standard definitions were required; • Maintaining DWAF’s M&E function whilst transforming itself from service delivery implementer to sector regulator; • Change management challenges (resistance to change; mutual trust and sector cooperation) • Institutional transformation challenges • Operational information acquisition from WSA level. • Sector collaboration on M&E&R is not mainstreamed and should be promoted to facilitate cooperative development of M&E&R frameworks between sector departments.

  24. 11. Conclusion • M&E&R is a combination of both quantitative and qualitative analysis of performance. • Greater emphasis needs to be placed on issues of operability, specifically in terms of post-construction M&E, with a focus on “value for money” and enhancement of QoL; • M&E&R frameworks need to be developmental in nature; • M&E&R sustainability dependent on consistency, accuracy and timeous information flow, providing different perspectives (operational, planning, strategic) to sector roleplayers; • Establishment of DWAF Regional M&E&R units will facilitate the development of regional sector M&E&R.

More Related