Monitoring and Evaluation

1. Interim Draft Module 11 - September 2008 Monitoring and Evaluation

2. Project Partners • Collaborative project Funded by the United States Agency for International Development (USAID)

3. Module Overview • Monitoring & Evaluation (M&E) framework and components • Records, registers and reports • Recording and evaluating response to TB treatment regimens International Standards 11 and 17

4. Learning Objectives At the end of this presentation, participants will be able to: • Describe what is meant by “Monitoring and Evaluation” • Discuss the importance of collecting data and ensuring the accuracy of the data • Explain ways in which the data are used to evaluate treatment • Describe how M&E activities can benefit both TB and HIV/AIDS programmes

5. Monitoring & Evaluation System • A key element of the DOTS Strategy • Allows programmes to: • Monitor progress and treatment outcomes of individual patients • Evaluate the overall performance of the TB programme at various levels (local, district, national) • Identify areas of programme improvement and weakness • Ensure accountability

6. Monitoring • What is it? • Routine tracking of services and programme performance

7. Monitoring (2) • How is it done? • Through information collection, data input, analyzing the data, and reporting what is found in that analysis outcome report • Why should we do it? • To better assess how well a policy or programme is achieving its intended target

8. Evaluation • What is it? • Episodic assessment of results that can be attributed to programme activities • Types of evaluation related to M & E: • Process evaluation: assesses the progress in programme implementation and coverage • Outcome and impact evaluation: measures the effect of the programme activity on the target population

9. What are the Targets? • Stop TB Partnership/WHO • 70% TB case detection and 85% cure rate by 2005 • Millennium Development Goals (MDG): • Halt, and begin to reverse, the incidence of major diseases such as HIV/AIDS and TB by 2015 • Decrease TB prevalence and death rates to 50% of the 2000 estimates • United Nations General Assembly Special Session (UNGASS) – global targets

10. Where do Indicators Fit In? • Indicator: a specific, observable, and measurable characteristic or change that shows the progress a programme is making toward achieving a specific outcome • Indicators may be expressed in terms of: • Number • Rate • Proportion • Percentage

11. Limitations of Indicators Indicators DO NOT: • Measure everything • Tell us WHY a problem may exist or HOW to fix it • Determine if problems identified are amenable to intervention • Tell us which interventions are most cost effective

12. Question What are some possible uses of data collected by the National HIV/AIDS and TB Programmes?

13. Using and Disseminating Data M & E can improve and enhance NAP and NTP work by: • Identifying areas of strengths and weaknesses • Helping plot progress toward programme goals • Allowing a programme to see trends and to identify high risk groups in order to better target TB control efforts • Providing justification for needed resources • Identifying training and supervision needs • Increasing public awareness about TB • Advocating for policy changes and allocation of funds

14. Monitoring & Evaluation Framework

15. Types of M & E Activities • Staff training • Supervision • Completion of reporting forms • Discussions during staff meetings • Ensuring medicine and laboratory stock supplies • Quality control activities • Analysing data and preparing reports

16. Recording & Reporting Standards • International Standards for TB Care #11 A written record of all medications given, bacteriologic response, and adverse reactions should be maintained for all patients

17. Recording & Reporting Standards (2) All providers must report both new and retreatment tuberculosis cases and their treatment outcomes to local public health authorities, in conformance with applicable legal requirements and policies • International Standards for TB Care #17

20. Question Why is accurate reporting and record keeping important?

21. Data Quality Assurance • Ensures that the information collected adequately represents the programme’s activities • Accurate data – measuring what it is intended to measure • Reliable data – collected and measured the same way by all programme personnel over time

22. Request for Sputum Examination Tuberculosis Treatment Card Tuberculosis Identity Card Basic Management Unit TB Register TB Laboratory Register Quarterly Report on TB Case Registration Quarterly Report on Sputum Conversion Register of TB Suspects Quarterly Report on Treatment Outcomes Yearly Report on Programme Management in Basic Management Unit Reporting Forms and Registers

23. Tuberculosis Treatment Card

24. Tuberculosis Treatment Card (2) (RHE) (Other) (RH)

25. Preparing a TB Treatment Card Activity

26. Tuberculosis Identity Card • This card contains an extract of information on the treatment card • It is given to the patient at the start of treatment • It is used to record daily DOT and must be used during the intensive phase of treatment • It also serves as a reference document for TB status after treatment • It should be presented to the doctor whenever the patient falls ill in the future

27. Tuberculosis Identity Card (2)

28. Basic Management Unit TB Register • This revised register is the cornerstone of an NTPs monitoring & evaluation system • It records essential information for notification & treatment outcome by district • It should always be kept up to date with data on sputum smear examinations and treatment outcome • Where electronic data collection systems are available, the information from the register should be entered into the database at least once every month

31. Treatment Outcomes • Cured* • Treatment completed* • Treatment failure • Died • Default • Transfer out *Treatment success – combines “cured” and “treatment completed”

32. Treatment Outcomes

33. Treatment Outcomes (2)

34. Supervision WHO, 2006

35. Question How is supervision used in your TB and HIV/AIDS Prevention and Control Programmes?

36. Role of Supervision in M & E • Supervision is a process of guiding, helping, training, and enabling staff to improve their performance in order to provide high quality health care services

37. Purpose of Supervision • Provide leadership and direction to staff • Ensure effective programme implementation • Monitor operations and evaluate achievement of goals • Ensure adherence to laws and policies • Avoid confusion or duplication of efforts

38. Purpose of Supervision (2) • Monitor that all necessary tasks are properly performed • Ensure that resources are properly used and are available to staff, including training and supplies to carry out their duties • Ensure accountability

39. Barriers to Effective Supervision • Lack of commitment • Lack of proper planning and time management • Lack of tools for Monitoring & Evaluation • Insufficient staff • Problems with transportation • Lack of confidence or preparation

40. Roles and Responsibilities • National Level – NTP and NAP • Planning, implementing, monitoring, and evaluating Programme at all levels • Regional/District/Parish Levels • Coordinating, supervising, planning, implementing, monitoring and evaluating all aspects of TB and HIV/AIDS Programmes in the region

41. How Can We Use M&E Information in Our TB and HIV Programmes? • Identify gaps in performance • Monitor treatment outcomes • Measure the impact of an intervention or policy change • Identify populations for enhanced control and prevention efforts • Identify local problems as they arise • Ensure high quality TB and HIV prevention and control strategies are consistently provided

42. Summary: ISTC Standards Covered* • Standard 11: A written record of all medications given, bacteriologic response, and adverse reactions should be maintained for all patients • Standard 17: All providers must report both new and retreatment tuberculosis cases and their treatment outcomes to local public health authorities, in conformance with applicable legal requirements and policies *[Abbreviated version}

43. Summary • Several approaches are used to monitor and evaluate TB and HIV/AIDS programmes including supervision, training and the keeping of records and registers • Accurate and timely reporting and record keeping is important. It allows true assessment of Programme achievements