Public Health Surveillance in a Changing Telecommunication Environment.

1. Public Health Surveillance in a Changing Telecommunication Environment. 21 January 2009 Ali H. Mokdad, Ph.D. Professor, Global Health

2. Outline • IHME • Background • Behavioral Risk Factor Surveillance System (BRFSS) • Improvements • Conclusion • Future

3. IHME Overview

4. Global Context: Attention and Opportunity • More media/political attention, social commitment and resources for global heath than ever before. • Political importance of health within countries and in foreign policy dialogue has increased. • Growing set of diagnostics, drugs, vaccines, procedures for improving health. • New willingness to experiment in how health resources should be channelled to those in need.

5. Why is IHME Needed Now? • To sustain interest in global health by demonstrating results; • To be more efficient by creating knowledge-related global public goods on what works and what doesn't that can form the basis for a process of shared learning among actors; • To promote the values of transparency and accountability worldwide; • To provide a strong foundation of publicly-available evidence that empowers policymakers, donors, practitioners, researchers, local and global decision-makers, and others to strategically allocate limited resources for measuring and ultimately improving population health.

6. IHME’s Core Funding Main sources: The Institute receives $2 million a year from Washington State These funds are supplemented with a 10 year grant of $105 million from the Bill and Melinda Gates Foundation. Project Grants: Projects are often funded by individual grants, rather than from the core grant or state money.

7. Vision and Goal Vision: IHME aspires to make available to the world high quality information on population health and its determinants, and on the performance of health systems. We seek to achieve this directly, by catalyzing the work of others, and by training researchers as well as policy makers. Goal: Our goal is to improve the health of the world’s populations by providing the best information on population health.

8. Who are we trying to reach? • Policy makers (elected, appointed, and technocrats) and policy professionals (staff, analysts, etc.) • Practitioners • Researchers in global health • Donors • General public

9. Executive Management Team Institute Director Director of Education and Training Director of Global Data Bank Director of Communications Director of Administration and Operations Director of Strategy and Special Projects

10. A Few Key Projects • AVAHAN Evaluation • GC13 • US Project • GBD Health Outcomes Health Services Resource Inputs Decision Analytics Evaluations Tools and Instruments Mortality Causes of Death Functional Health Status Burden of Disease and Comparative Risk Assessment Effective Coverage Common Indicators Inequalities Evaluation Methods Evaluation Studies International Resources for Health National Public Expenditure Models National Health Information Systems Technical Quality Responsiveness Forecasting Priority Setting Health Systems Performance Assessment Household Expenditure Human Resources Survey Design and Implementation AVAHAN GC13 US Project GBD Active Dormant

11. Staff 110 70 54 40 4 June ‘09 July ‘07 July ‘08 Oct ‘08 2011

12. Tracking progress towards universal childhood immunisation and the impact of global initiatives:a systematic analysis of three-dose diphtheria, tetanus, and pertussis immunisation coverage Stephen S Lim, David B Stein, Alexandra Charrow, Christopher JL Murray

13. Background Two main questions: • What is the trend in three-dose diptheria, tetanus and pertussis vaccination (DTP3) coverage based on surveys over the period 1986 to 2006? • Do target-oriented initiatives such as universal childhood immunization (UCI) and results-based financing initiatives such as GAVI’s Immunization Services Support (ISS) lead to over-reporting of DTP3 immunization coverage?

14. GAVI Immunization Services Support (ISS) • Performance-based payment • Number of additional children reported by countries to have received DTP3 • Reports largely based on administrative data • Baseline is the year prior to approval of the proposal • US$20 is paid once per additional child • Data quality audit (DQA) of administrative data system before reward payments commence (from Year 3)

19. Global trends in DTP3 coverage Survey-based global coverage of DTP3 (black) with 95% uncertainty estimates compared to countries’ officially reported (red) and WHO and UNICEF estimates (blue), 1986 to 2006.

20. GAVI Immunization Services Support (ISS) • Number of additional children vaccinated in 51 countries receiving ISS funding up to the year 2006 : • Based on official reports: 13.9 million • Survey-based: 7.3 (5.5 to 9.2) million • ISS payments • Based on official reports: US$289 million • Survey-based: US$148 million

21. GAVI Immunization Services Support (ISS) • Out of 51 countries • 4 countries that reported increases, number of additional children did not increase • 6 overestimated by > 4x • 10 overestimated by > 2x but ≤ 4x • 23 overestimated by > 1x but ≤ 2x • 8 countries underestimated

22. Summary • Substantial resources are being directed towards increasing the effective coverage of interventions to improve population health • Must ensure that increased resources for health are being utilized cost-effectively and for their intended purpose • Independent, contestable, empirically-based monitoring of health indicators

23. Challenges in Public Health SurveillanceBackground

24. Coverage problems • Face-to-face coverage: • Available household lists not complete • Need to manually count and list • Telephone coverage: • Households with no telephones (2-3%) • Cell phone only households (13-15%) • No directory of cell phone numbers • Number portability and erosion of geographic specificity • Mail coverage: • USPS list only readily available source for general populations • Poor coverage in rural areas • Email coverage: • No systematic directory of addresses

25. Declining response rates • Response rates decreasing significantly in the last 10 – 15 years. • Decline has occurred for most types of surveys— particularly telephone and in-person interviews • Evidence of trends for mail surveys not as clear due to lack of long-term trend studies. • Web surveys are too new to provide good trend data. • Increase in nonresponse is a global problem • No single or clear explanation for these trends.

26. Behavioral Risk Factor Surveillance System (BRFSS) • Monthly state-based RDD survey of health issues • 50 states, District of Columbia, Puerto Rico, Guam, and Virgin Islands • 430,000+ adult interviews conducted in 2007 • From 2002 to 2007: • completed 1,950,000 interviews • Dialed 18,500,000 telephone numbers

27. BRFSS Strengths • Flexible • Timely • Standardized • Useful

28. Prevalence of Obesity* Among U.S. Adults (*BMI 30, or about 30 lbs overweight for 5’4” person) 1996 1990 2004 No Data <10% 10%–14% 15%–19% 20%-24% 25% Prevalence of Diabetes* Among U.S. Adults (*Includes gestational diabetes) 1990 1996 2004 No Data <4% 4%-6% 6-8% 8-10% >10%

29. Prevalence of Women Who Never Had a Mammogram, Ages 40 and Older BRFSS 1990–2004

30. Support Policies and Legislation:Mandatory Insurance Coverage for Screening Mammography 1981 1990 2004 No mandatory insurance coverage for screening mammography. Mandatory insurance coverage for screening mammography. Source: National Cancer Institute — State Cancer Legislative Database Program, Bethesda, MD, 2004.

31. Support Policies and Legislation:Prevalence of Safety Belt Use, 2002 Areas with primary safety belt laws Prevalence > 80% of always using a safety belt among persons aged > 18 years. Prevalence < 80% of always using a safety belt among persons aged > 18 years. Source: CDC. Impact of primary laws on adult use of safety belts – United States, 2002. MMWR 2004;53:257-260.

32. Develop Local Programs and Policies: SMART BRFSS in Fargo • Fargo, ND – 24.9% binge drinking vs. 16.4% nationwide • Formed community coalition: AMP (Alcohol Misuse Prevention) • Mission: Reduce alcohol use among those under 21 in the Fargo-Moorhead area. • Anti-binge drinking campaign • Policy change sanctioning facilities • Intervention with ER doctors

33. People Prepared for Emerging Health Threats

34. Vaccine Shortage – Timeline • Oct 5: Vaccine shortage announced • Oct 5: Initial discussions within CDC • Oct 19: Call with BRFSS state coordinators • Oct 19-26: New questions developed and cognitively tested • Oct 27: CATI specifications to states • Nov 1: Data collection began

35. December MMWR • Dec 1-11: States collected December data • Dec 13: Submitted files to CDC • Dec 16: Dr. Gerberding holds press conference & MMWR released on the CDC website

36. Response rate trends: Behavioral Risk Factor Surveillance System (BRFSS)

37. Percentage of U.S. HouseholdsWithout Landline Telephones • Based on National Health Interview Survey data

39. Percent Distribution of Household Telephone Status for Adults, July-December 2007 Phoneless: 1.9% Unknown: 1.3% Wireless Only: 14.5% Landline Only: 19.1% Wireless Mostly: 14.0% Landline with Some Wireless: 49.2%

41. Coverage • Sampling frame must include all units of the population of interest • Frame coverage errors • Coverage by key modes: • Face-to-face: expensive to develop (lists help) • Telephone: cell phones and number portability • Mail: USPS list not complete, but improving • Web: no comprehensive list

42. Sampling • Each element has a known and non-zero probability of selection from sampling frame • Protection against selection bias • Quantify sampling error • Sampling by key modes: • Face-to-face and telephone: well developed techniques • Mail: within household selection techniques quasi-random for general pop surveys • Email: tend to be nonprobability; opt-in samples

43. Nonresponse • Inability to obtain data from selected respondent: • Unit nonresponse • Item nonresponse • Nonresponse by key modes: • Highest in face-to-face • Lowest in internet • Item nonresponse varies by mode and question

44. Measurement • Measurement error occurs when a respondent’s answer to a question is inaccurate (departs from the “true” value) • Modes vary in terms of: • Interviewer versus self-administered • Stimuli / manner in which survey question is conveyed to respondent (and response is recorded)

45. Interviewer administered questions • Help to: • Motivate respondents • Guide through complex questionnaires • Clarify questions and instructions • Probe for detailed answers • Potential problems: • Less privacy, less anonymous • Social desirability • More positive

46. Self-administered questions • Help to: • Ensure privacy • Self-paced • Conduct survey at convenience of respondent • Potential problems: • Ensuring correct respondent completes survey • Little/no option for requesting assistance • Stray/out-of-range responses • No means of assess cognitive engagement of respondent

47. Goal: Optimize survey design to decrease total survey error for a given cost Coverage Sampling Costs Nonresponse Measurement

48. Costs for single modes • Face-to-face (most expensive): • 5-10 times higher than telephone • Telephone: • 2-3 times more expensive than mail • Mail: • higher than Web due to fixed costs plus per interview processing • Web (least expensive): • Primarily fixed set-up costs, little per interview costs

49. Assessing Data Validity

50. What do we mean by “validity”? • The closeness of our survey estimates to the “true value” • Ideally there is no difference • Potential survey bias is minimized • “Bias” in survey estimates results from product of: • Level of nonresponse • Difference between respondents and nonrespondents on measures of interest