Unit 9 : Evaluating a Public Health Surveillance System

Unit 9: Evaluating a Public Health Surveillance System # 1-9-1

Warm Up Questions: Instructions • Take five minutes now to try the Unit 9 warm up questions in your manual. • Please do not compare answers with other participants. • Your answers will not be collected or graded. • We will review your answers at the end of the unit. # 1-9-2

What You Will Learn • By the end of this unit you should be able to: • list tasks for evaluating a surveillance system • develop a plan for evaluating your own country’s surveillance system # 1-9-3

Why Evaluate? • Evaluation ensures that your HIV/AIDS surveillance system remains effective as the epidemic shifts over time. • If your system is no longer effective, you will not have the right information to control HIV/AIDS. # 1-9-4

Purpose of Evaluation • Appraise and prioritise the disease events to be kept under surveillance • Assess how the system can detect and report these diseases • Assess the quality of the epidemiologic information produced # 1-9-5

Purpose of Evaluation, Cont. • Assess how the system can respond to these diseases • Assess how surveillance results affect disease control and policy • Identify which elements of the system can be enhanced in order to improve the quality of information # 1-9-6

Evaluation • Document current state of the surveillance system • Identify strengths and weak points • Recommend improvements • Define training requirements or gaps • Justify resources • Additional funding • More and better training • Improved surveillance for better disease control possible outcomes Figure 9.1. Elements of an Evaluation # 1-9-7

Six Evaluation Tasks • Engage the stakeholders in the evaluation. • Describe the surveillance system to be evaluated. • Focus the evaluation design. • Gather credible evidence regarding the performance of the surveillance system. # 1-9-8

Six Evaluation Tasks, Cont. • Justify and state conclusions and make recommendations. • Ensure use of evaluation findings and share lessons learned. For more detail, refer to the Updated Guidelines for Evaluating Public Health Surveillance Systems (Centers for Disease Control and Prevention, 2001), available at www.cdc.gov/mmwr/preview/mmwrhtml/rr5013a1.htm # 1-9-9

Task 1: Engaging Stakeholders in the Evaluation • Stakeholders include: • Public health practitioners • Healthcare providers • Data providers and users • Community representatives • Governments at the district, provincial and national levels • Governmental, non-governmental, professional and private non-profit organisations # 1-9-10

Task 1: Engaging Stakeholders in the Evaluation, Cont. • Examples of ways to engage stakeholders include: • Hold a community meeting to discuss plans for the evaluation. • Hold one-on-one meetings with the key people listed above. • Invite participants to join the evaluation team. # 1-9-11

Class Discussion: Task 1 • Task 1 is to ‘engage stakeholders.’ • Who would this be, in your case? • Why is this task important? # 1-9-12

Task 2: Describe the Surveillance System to be Evaluated • The evaluation should describe: • The public health importance of the health-related event under surveillance • The purpose and operation of the system • The resources used to operate the system # 1-9-13

Class Discussion: Task 2 • Task 2 is to ‘describe the surveillance system.’ • In your case, where would you find the type of information listed in the task? # 1-9-14

Task 3: Focus the Evaluation Design • Determine the specific purpose of the evaluation. • Identify stakeholders who will receive findings. • Consider what to do with the information generated from the evaluation. • Specify the questions that will be answered. • Determine the standards for assessing the performance of the system. # 1-9-15

Class Discussion: Task 3 • For task 3: • What are some ways you can think of to design an evaluation? • How would you know what to evaluate? # 1-9-16

Simplicity Flexibility Data quality Acceptability Positive predictive value Representativeness Timeliness Stability Data use/ dissemination Task 4: Gather Credible Evidence Regarding the Performance of the Surveillance System Describe the surveillance system in terms of its: # 1-9-17

Class Discussion: Task 4 • Task 4 concerns gathering data. • Should there be a time limit on gathering data? • What would you do if you found that some clinics didn’t have data or had incomplete data? # 1-9-18

Task 5: State Conclusions and Make Recommendations • Conclusions can be justified through appropriate analysis, synthesis, interpretation and judgment of the gathered evidence. • Recommendations should address modifications and continuations of the public health surveillance system. # 1-9-19

Class Discussion: Task 5 • In task 5, you are asked to make recommendations. • How is this typically done in your district or region? # 1-9-20

Task 6: Ensure Use of Findings and Share Lessons Learned • Develop strategies for communicating the findings from the evaluation. • Tailor recommendations to relevant audiences. • Recommendations for improvements should be distributed to all partners and sites involved in sentinel surveillance. # 1-9-21

Class Discussion: Task 6 • Task 6 concerns communications. • How would you decide whom to send your final report to? • In your case, would the report go to subject matter experts, perhaps Ministry of Health HIV/AIDS experts? • If not, how could you make sure that the people reading the report would understand what you were saying? # 1-9-22

In Summary • Evaluating your HIV/AIDS surveillance system is important, to ensure that remains effective as the epidemic changes over time. • The tasks that you should implement during this evaluation include: • engaging stakeholders • describing the surveillance system • focusing the evaluation design • gathering evidence on the system’s performance • stating conclusions and recommendations • sharing the lessons learned # 1-9-23

Warm Up Review • Take a few minutes now to look back at your answers to the warm up questions at the beginning of the unit. • Make any changes you want to. • We will discuss the questions and answers in a few minutes. # 1-9-24

Answers to Warm Up Questions 1. List three stakeholder groups that should be engaged during the evaluation of the surveillance system. # 1-9-25

Answers to Warm Up Questions 1. List three stakeholder groups that should be engaged during the evaluation of the surveillance system. • public health practitioners • healthcare providers • government officials • representatives of affected communities • non-profit and donor organizations • etc. # 1-9-26

Answers to Warm Up Questions, Cont. 2. If there is a high probability that cases identified by the surveillance system are actually cases of HIV infection, the system is said to have high: a. sensitivity b. representativeness c. acceptability d. positive predictive value # 1-9-27

Answers to Warm Up Questions, Cont. 2. If there is a high probability that cases identified by the surveillance system are actually cases of HIV infection, the system is said to have high: a. sensitivity b. representativeness c. acceptability d. positive predictive value # 1-9-28

Small Group Discussion: Instructions • Get into small groups to discuss these questions. • Choose a speaker for your group who will report back to the class. # 1-9-29

Small Group Reports • Select one member from your group to present your answers. • Discuss with the rest of the class. #1-9-30

Case Study: Instructions • Try this case study individually. • We’ll discuss the answers in class. # 1-9-31

Case Study Review • Follow along as we go over the case study in class. • Discuss your answers with the rest of the class. #1-9-32

Questions, Process Check • Do you have any questions on the information we just covered? • Are you happy with how we worked on Unit 9? • Do you want to try something different that will help the group? # 1-9-33

Unit 9 : Evaluating a Public Health Surveillance System