Monitoring, Evaluation & Technical Assistance

1. Monitoring, Evaluation & Technical Assistance

2. Session Goals • Identify key entry points for TA • Demonstrate linkages between M&E, procurement • and finance • Reemphasize the importance of M&E in the PBF • model.

3. Proposal development Entry Points for TA in M&E • M&E Plan • Attachment A / Performance Framework • Grant negotiation • Grant implementation

4. M&E Plan vs the Performance Framework The Performance Framework The Performance Framework is a legally-binding document attached to the grant agreement. It mainly focuses on a subset of indicators from the M&E plan which are used to measure the program’s performance and consequently inform disbursement decisions. The M&E Plan An M&E plan is typically a document developed in consultation with all major stakeholders that describes how the national (or Global Fund grant-specific) M&E system works and the costed actions needed to strengthen it over a set period of time.

5. Features of a “good” M&E Plan

6. What M&E Plan to submit? Normally PR should submit the National M&E plan, unless: Multi-country proposal  Regional M&E Plan National M&E Plan does  National M&E Plan with not include details GF-specificannex required for GF grant No National M&E Plan  Submission of GF-specific M&E plan and subsequent elaboration of National M&E plan

7. Budget – 5-10% Action plan Data quality assurance mechanisms Description of how system functions Impact / outcome framework Most Common Pitfalls with M&E Plans

8. Ensure sufficient budget allocation for M&E and for TA Break down M&E costs by cost category Show GF contribution to national M&E agenda and budget (vis a vis the gap) M&E Budget

9. M&E Personnel M&E Technical assistance Training, workshop and meetings Data quality assurance Equipment, infrastructure (including IT) and consumables/running costs Production of M&E Plans, work plans, guidelines, tools and reports Surveys, surveillance, operational research and special studies M&E Budget Cost Categories

10. Impact / outcome framework Baselines Target setting TA for the Performance Framework

11. Clear description of national I/O agenda Aligned to national priorities, targets and plans Sufficient budget allocation for measurement Show contribution from other partners and demonstrate funding gap Impact / Outcome Framework

12. “The TRP found the overall quality of the performance frameworks presented as poor. Monitoring and evaluation (M&E) capacity is still weak and predominantly dependent on process rather than outcome and impact indicators.” Quote from Technical Review Panel

13. Ensure updated baselines used in proposals Provide solid estimates for target populations For baselines and reporting on routine monitoring, provide number and % Baselines

14. Ensure targets are in line with overall goals Based on epidemiological situation, past performance, capacity in country, capacity for scale up etc. Value for money Clear link with finance and procurement Target Setting

15. Ensurealignment to national strategy’s objectives, indicators and targets. Support the submission of the national M&E plan every time a grant specific M&E plan is not needed. Key messages

16. Proposal development Grant negotiation Grant implementation Entry Points for TA in M&E

17. M&E systems strengthening workshops Development of M&E strengthening action plans Development of M&E plans Assistance with performance frameworks TA Entry Points during Grant Negotiation

18. Proposal development Grant negotiation Grant implementation Entry Points for TA in M&E

19. Address issues identified ad-hoc, through DQA or OSDV Implementation of M&E system strengthening action plan TA during Grant Implementation

20. Systems Training of people Operational definitions (of indicators) The Most Common M&E Issues where TA Could Help during implementation

21. Vital role of TA at all stages of Global Fund grants Importance of M&E in the performance based funding model Be honest and realistic with your clients Final Thoughts

22. Group Exercise

23. Goal: To understand, through a real world example, the interlinkages for TA between M&E, procurement and finance Groups of 5 or 6 Methodology: Group-based problem solving and development of TA action plan. Expected Outputs: Diagnosis of problem Identification of potential solutions TA action plan from M&E perspective Exercise Instructions

25. Extra Slides It would be helpful to prepare an introduction to each session (1 slide) explaining: The methodology of the session (presentation – Q&A – exercise) Set of questions. Number of groups for the exercise. Expected output. The introduction format should be the same for the three presentations, a common methodology, common tool, with different specific results (and questions). The common output could represent a better understanding of the links and interdependence between the different areas that characterize a proposal (or technical assistance).

26. List of indicators as agreed upon in the national strategy For each indicator: Indicator name Baseline values with dates and relevant source of data Targets set according to frequency of measurement Data collection method for the indicator – e.g. health information system, programme monitoring, ANC sentinel surveillance, population-based surveys, facility-based surveys, etc. Frequency of data collection (monthly, quarterly, annual, 3 years, etc.) Person/agency responsible for data collection and reporting (if agency, please specify relevant unit) M&E Framework & Data collection methods (required)

27. Information products (required) • The M&E plan should explain how the data collected and analyzed will be made available to stakeholders and the general public. • E.g.: periodic reports, websites, • stakeholders’ meetings, etc.

28. The M&E plan should contain a data verification and quality assurance system The Global Fund has developed a system of on-site data verification that could be expanded beyond Global Fund grants Data quality assurance (required)

29. The M&E plan should include an Action Plan which includes activities, time of implementation, responsible unit and budget Should be developed and agreed upon among different stakeholders (especially those involved in the implementation and support of the activities included) Can be developed with M&E System Strengthening tool which should be considered a living document, constantly monitored and to be reviewed and updated on a regular basis. Action Plan (required)

30. The M&E plan should contain an M&E budget with sources of funding and financial gaps Global Fund contribution should be indicated 5-10% of grant is recommended to be used to strengthen the M&E system Examples of budget items are: HMIS strengthening, TA, workshops, surveys, staff, information products, etc. M&E Budget (required)

31. Operational research (OR) Evaluation Other types of studies that complement the information collected through the indicators and will periodically evaluate the programme and M&E system Evaluation and OR (desirable)

32. how the data is received, stored & analyzed (specify electronic database - if online, provide link) and accessed by relevant actors data flow from sources to final users Data management (desirable)

33. The M&E plan should include a description of the capacity present at the time of developing the plan capacity gaps strategy adopted to improve the M&E capacity over the plan’s lifespan Capacity building (desirable)

34. The M&E plan should include: the agencies and units responsible to run the M&E system and implement the M&E plan the relevant M&E units at national and, if they exist, sub-national levels recommendation to include a multi-sector M&E Technical Working Group Coordination (desirable)

35. Herramienta de Fortalecimiento de los Sistemas de Monitoreo y Evaluación

36. Qué es la Herramienta de Fortalecimiento de los Sistemas de Monitoreo y Evaluación? Paso 1:Diagnóstico Paso 2: Plan de Acción

37. La Herramienta de Fortalecimiento de los Sistemas de M&E • Una herramienta acordada con nuestros socios técnicos (pronto será reemplazada con una sola herramienta con 12 componentes) • Cubre tres listas de comprobación complementarias: • Plan de M&E • Capacidades de gestión de la información de la Unidad de Gestión (unidad de Monitoreo y Evaluación) • Sistemas de Notificación de Datos por Área de Programa (e.g., ARV, DOTS)

38. Los resultados que se esperan son: • Un plan de acción detallado con el presupuesto para fortalecer los sistemas de M&E • Una discusión mas amplia durante la negociación de la subvención y/o durante la implementación de la subvención. • Identificación de las necesidades que requieren apoyo técnico (y el proveedor de este apoyo) • Un acuerdo con los socios técnicos del país sobre los iniciativas compartidas y su financiamiento (p.e., la organización de encuestas)

39. Evaluar el sistema de M&E Desarrollar/revisar el Plan de M&E y Plan de Acción y Fortalecimiento Implementar el Plan de M&E y Plan de Acción y Fortalecimiento Dar Seguimiento El Ciclo de Fortalecimiento de M&E