Government-wide monitoring and evaluation system (GWM&E)

1. Government-wide monitoring and evaluation system (GWM&E) Presentation to the Public Service and Administration Portfolio Committee 18 October 06

2. Presentation outline • Aim of GWM&E • GWM&E Results • Implementation phases • Management structures • Achievements

3. Government-wide monitoring and evaluation system (GWM&E) • The aim of the GWM&E system is to contribute to improved governance, promote learning and enhance the effectiveness of public sector organisations and institutions and accountability reporting • The system objectives are the collection and collation, analysis and dissemination and the application of information on the progress and impact of programmes in order to: • Ensure transparency and accountability • Promote service delivery improvement • Ensure compliance with statutory and other requirements and • Promote the emergence of a learning culture in the public sector

4. Results to be achieved • Result one: Accurate and reliable information on progress in the implementation of government and other public sector programmes are collected and updated on an ongoing basis • Result two: Information on the outputs, outcomes and impact achieved by government and other public bodies are periodically collected and presented • Result three: The quality of monitoring and evaluation practices in government and public bodies is continuously improved

5. GWM&ES Proposal and Schedule & Implementation Plan • Phase One: Setting the basis for implementing the reporting practice • Development of Principles & Practices • Compendium of Development Indicators for SA • OPSC assessment of departmental M&E Systems • Capacity Building and Accreditation • Phase Two: System Functionality, Alignment and Standardisation. • Drafting, consultation & finalization of data gathering templates and reporting formats • Develop ICT platform & architecture to receive, collate, and report on data collected. • Ensuring the interoperability and integration between this ICT platform (GWM&ES) and existing departmental ICT systems.

6. GWM&ES Proposal and Schedule & Implementation Plan • Phase Three: System Functionality, Alignment and Standardisation. • Initiate a pilot with a few departments to test system functionality. • Develop a detailed project plan to serve as a template for later roll-out. • Independent evaluation of the pilot system roll-out using the evaluator's findings and recommendations to make system adjustments. • Gradual extension of the system to all government bodies,starting with national departments, then provincial departments,local authorities, and all SOEs.

7. GWM&ES Proposal and Schedule & Implementation Plan Phase One: Setting the basis for implementing the reporting practice 1. Development of Principles & Practices • What are principles and practices? • Government is adopting an implementation approach based on shared principles and common practices • This is a different strategy to one based on rules and prescriptions (old-style public administration) • Basing our emergent system on principles provides a clear framework while it encourages adaptive thinking, consultation and communication

8. Purpose of M&E Principles & Practices • The document sets out the national M&E principles and practices, including minimum norms and standards for monitoring and evaluating government policies, projects and programmes. • It also focuses on procedures to be adopted by national, provincial, local and public entities in the monitoring and evaluation of implementation of government policies and programmes/projects

9. Core principles • Principles are the unchanging, non-negotiable values that underlie the system • Although proposed, the GWM&ES’s core principles have not yet been determined • The system will have different kinds of principles at its heart:

10. Types of principles The GWM&ES must be built on: • Ethical principles: Rights-based; transparent and accountable • Managerial principles: A balance of efficiency, economy, effectiveness; service delivery oriented • Technical principles: Indicator based, participatory, people-centered, developmental, inclusive etc. • Proprietary Principles: are intended to ensure that an evaluation will be conducted legally, ethically, and with due regard for the welfare of those involved in the evaluation, as well as those affected by its results • Feasibility Principles: The feasibility principles are intended to ensure that an evaluation will be realistic, prudent, diplomatic and frugal; The evaluation procedures should be practical, to keep disruption to a minimum while needed information is obtained; Be cost effective

11. GWM&ES Proposal and Schedule & Implementation Plan Phase One: Setting the basis for implementing the reporting practice 1. Development of Principles & Standards • Milestones Achieved • Initial Draft completed: May 2006 • Workstream established : June 2006 • TORs for the work stream approved by M&E Task Team: July 2006 • Conference to deliberate on the matter held: 19-23 June 2005 • Comments/Input from conference incorporated: July 2006 • Consultations with Relevant Stakeholders completed : August 2006 • Ongoing progress on the activity reported to Cluster & Cabinet via POA: Bi-monthly

12. GWM&ES Proposal and Schedule & Implementation Plan Phase One: Setting the basis for implementing the reporting practice 1. Development of Principles & Practices (including Norms & Standards) • Milestones Achieved • Draft Guidelines for M&E being discussed at Work Stream level • Key Glossary of M&E Terms completed August 2006

13. GWM&ES Proposal and Schedule & Implementation Plan Phase One: Setting the basis for implementing the reporting practice 2. Compendium of development Indicators for SA • Milestones Achieved • Based on the Ten Year Review, appropriate set of the indicators for five clusters have been defined, which comprise of 163 indicators and a total of 335 subset indicators that would be used to measure the impact of government performance in the long term. • The matrix constitutes eight significant variables namely: • Index (Category); Index Description; Indicator; Subset Indicators; frequency of data collection; data source; data Sets and Level of disaggregation • Deliberated upon at a workshop in November 2005 & Conference in June 2006 • Indicator refinement & definitions by PCAS/NSSD: Ongoing work • Discussed at Cluster Level: January/March 2006 • Draft baseline document completed & circulated for comments: May & Sept 2006 • Baseline data collection: completed Sept- 2006 (for quantitative indicators)

14. GWM&ES Proposal and Schedule & Implementation Plan Phase One: Setting the basis for implementing the reporting practice 2. Compendium of development Indicators for SA (Cont) • Milestones Achieved • Data has been collected and has been captured in several computer packages (MS Access, MS Excel, MS Word and SPSS) • Integrated all the available databases into one usable database - DevInfo in order to achieve a: • a common understanding of indicators programme • Analysis, assessment and evaluation of the validity and reliability of SA information • Ability to collate, disseminate and present both government, millennium development goals and other international commitments • The software is available to SADC countries at no cost

15. GWM&ES Proposal and Schedule & Implementation Plan Phase One: Setting the basis for implementing the reporting practice 3. OPSC assessment of departmental M&E Systems • Description • The OPSC was requested to undertake an audit of departmental M&E systems and reporting requirements. The purpose of this report is to provide an assessment of what exists in terms of M&E at a departmental level and an understanding of how departments undertake their reporting requirements. • The report is currently being finalised and will be presented to various stakeholders soon. • In the context of the above, it should be noted that the PSC is implementing its governance monitoring system across departments. To-date over 35 such reports have been completed.

16. GWM&ES Proposal and Schedule & Implementation Plan • Phase One: Setting the basis for implementing the reporting practice • 4. Capacity Building and Accreditation • Approach • One core training manual will be developed (based on the Oxfam model) in the GWM&ES, the training will be based on the system in particular, not just on M&E in general. • Adopting this approach will ensure that many self proclaimed M&E experts will not be allowed to market their training as “GWM&ES training”, this will be done for government, by government agencies. • Different facilitation guides will ensure that the level and complexity target beneficiaries will be addressed, in other words, if an M&E manager is being trained, the complexity and depth of the training will be commensurate with the M&E responsibilities that she has to fulfill. The same applies to frontline staff who have very basic M&E responsibilities, for instance, recording information on client contact or services rendered.

17. The Proposed Capacity Building Strategy Assessment of M&E capacity needs at provincial level • Does the province have an M&E Strategy that is clearly articulated? • Has the province conducted departmental M&E Audits? • Does the province have an M&E Strategy that is clearly articulated? • Does the province have an M&E Learning Network? • What provincial M&E capacity building initiatives have taken place thus far? • Is the provincial M&E Strategy clearly aligned with principles and standards of the GWM&ES? • Does the provincial M&E Strategy accommodate cluster reporting • Does the provincial M&E Strategy provide useful information on the Service Delivery Objectives of the Office of the Premiere? • Has the province launched its IDP nerve centre?

18. Conceptual learning framework - capacity building consist of three dimensions Training and development • Competency-based training - Knowledge, skills and values • Additional seminars/ workshops more detail eg Advanced Statistics • Accredited training to be recognised at tertiary institutions further studies • Morning sessions for Chief directors and above – advocacy for GWM&E Organisation development • Consultation and support with development of systems, data and reporting • Change management Community of practice • Principles and practices, learning networks to support. • SAMEA to support the M&E Forum for Government officials and recognition for officials declared competent on training

19. Development of training material • Focus group engagement to determine learning outcomes / topical issues and methodology as design framework for training material • Consultation workshop with Premiers’ offices and key stakeholders on the Curriculum framework • Approval from M&E task team and SAMDI Curriculum Review Committee • Submit funding proposal for development of training material (and implementation in key departments) • One training manual – differentiate through facilitators guidelines and target groups identified (Application of learning; exercises; action plans) • Appointment of persons (internal or external) to develop training material and obtain approval from Curriculum Committee. • Quality Assurance and Research on capacity development to enhance training and development adding value to relevance and implementation.

20. GWM&ES Proposal and Schedule & Implementation Plan Phase Two: System Functionality, Alignment and Standardisation. 1. Drafting, consultation & finalization of reporting formats • Milestones Achieved • The joint team of NT and DPSA have commenced with the development of the requirements of a system that would ensure the monitoring of the departmental strategic plans – The requirements will be finalised by the end of October 2006 • The work stream has commenced with the development of the framework of an end of year report that would complement the current report with information currently within the transversal systems – The framework will be finalised at the end of November 2006

21. Annual State of the Nation Report(End of year report) • This report will inform/support the January Lekgotla and the SONA • Content: • Cluster annual progress reports based on POA • Combination of transversal systems information • Aggregation of departmental strategic plans, annual reports and departmental individual performance plans • The reporting format, indicators, collection & reporting system are still being developed

22. Cluster Performance Strategic Plan Human Resource Batho Pele Outcome/Impact Admin & Manag Financial Outputs Sector Performance Dept/Prov Performance Annual Reports & Quarterly Rep. Reporting Requirements Reporting Formats 2009 HR Utilisation Public Man Watch OPSC GWM&E Database National Treasury DPLG POA

23. Current Transversal M&E Systems

24. Public Service M&E System – (OPSC) • SA Constitution lists 9 basic values and principles that govern the public service • The PS M&E System looks at the extent to which public service departments comply with these principles • The system aims to • Gather info that can be used to identify areas needing attention, ultimately contributing to improved public service performance • This information is published on an annual basis in the State of the Public Service Report

25. Cluster progress – Presidency • This system is based on the matrix of indicators which is a set of indicators for each of the clusters • The aim of the system is to track the progress of the different clusters in reaching their outcomes • The system indicates • Those areas where progress is slow and intervention is needed • Those areas of good progress as examples of best practices • This system is supported by the NSS and reports are updated on a quarterly basis

26. Programme of Action (Presidency) • The system is based on the Government Programme of Action that flows from the President’s State of the Nation Address that provides the strategic direction for the Public Service • The aim of the system is to provide an indication for progress in implementing the priorities of government. • The system indicates • Those areas where progress is slow and intervention is needed • Those areas of good progress as examples of best practices • The system is electronic database Excel templates and are updated every quarter with a two-month lag per quarter as due date for reporting • Feedback on progress is given on a quarterly basis by means of consolidated reports on the intranet of Treasury once the quality of the data is credible it will be available on the internet

27. In Year Monitoring (National Treasury) • Monthly monitoring reports: • Focus on performance against budget • Alert managers where remedial action is required • Reports consolidated and published: • National Departments – monthly publication in the national Government Gazette • Provinces – Monthly reporting and monitoring BUT quarterly publication in the national Government Gazette • Section 32 Reports

28. Non Financial Reporting (National Treasury) • This system is based on provincial departments’ reporting against targets set in their Annual Performance / Strategic Plans • The provincial indicators are set per sector to enable cross- provincial comparison • The system indicates • Those areas where progress is slow and intervention is needed • Those areas of good progress as examples of best practices • This system together with the In Year Monitoring System would facilitate cost benefit analysis • The system is updated on a quarterly basis • The system has been rolled out but there is still a problem with the quality of the data provided by departments

29. Public Management Watch (DPSA) • Early warning type system with regard to management within departments • Consists of the following processes: • Statistical rating based on payroll and financial information • Communication to and self assessment by identified departments • On site assessment of selected department • Intervention into the management of departments • The system is functional and will be applied every quarter • The system - • Indicates those departments/areas in need of intervention • Indicates those departments that have proven to be examples of best practices • Facilitate the knowledge exchange between departments • This system aims to direct the interventions of the centre of government to those departments that require assistance

30. Annual HR Report (DPSA) • Annual analysis of the status of HR in the Public Service • The following would serve as information sources for this report • Departmental annual reports • Public Management Watch findings • Payroll and Financial Systems • Reports – OPSC, AG, DPSA, etc. • The system - • Indicates trends within the HR environment • Identifies those bottlenecks effecting the efficient and effective utilisation of HR • Identifies those remedial steps taken to address the bottlenecks between departments • This system aims to guide the centre of government and HR managers on the areas in need of attention

31. GWM&ES Proposal and Schedule & Implementation Plan Phase Two: System Functionality, Alignment and Standardisation. 2. Develop ICT platform & architecture to receive, collate, and report on data collected • Milestones Achieved • The development of the IT system is dependent upon the finalisation of the user requirements (data and reporting requirements)

32. GWM&E – Proposed reporting outputs • Bi-monthly reports against the government Programme of Action (POA) • Mid-year progress report • Annual State of the Nation Report (End of year report) • Term of office PS Annual report • 2014 – Review • Special sector M&E reports – eg Education and Health

33. Management structure of the Institutional Framework for Implementing the GWM&ES Governance & Administration Cluster The Presidency Intellectual Leadership & Strategic Direction Project Steering Committee Project Management, Systemic Gearing and Capacity Building Project Implementation Tools, Standards, Templates, Platforms, Training Interventions M&E Task Team Capacity Building Work Stream guided by a TOR Principles and Standards Work Stream guided by a TOR Reporting and Databases Work Stream guided by a TOR

34. Institutions Responsible for Implementing Work Streams Reporting and Databases Capacity Building Principles and Practices Lead Agency Presidency DPSA SAMDI Partner Agencies OPSC Presidency Presidency Stats SA Stats SA Stats SA National Treasury SITA DPSA DPLG DPLG DPSA National Treasury

35. Overall Achievements to date • A public commitment to creating the system is an important first step and is shown in Cabinet’s adoption of the Strategy and Implementation Plan • Political support (key to public service reform at this level) has been consistently provided • Consistent, sustained membership and involvement of core task team members • Recent agreements on appointment of task teams and identification of members to facilitate reporting on the programme of action 2006 • POA system is fully functional and promotes transparency and accountability • Indicator development • Transversal systems have been developed and are operational