An Insider’s View of the US Preventive Services Task Force

1. An Insider’s View of the US Preventive Services Task Force George F. Sawaya, MD Associate Professor Department of Obstetrics, Gynecology and Reproductive Sciences Department of Epidemiology and Biostatistics University of California, San Francisco Director, Cervical Dysplasia Clinic, San Francisco General Hospital

2. History of the Task Forces • 1976 - Canadian Task Force on PHE • 1984 - USPSTF established by PHS • 1996 – Community Task Force • 1998 - 3rd USPSTF reconvened by AHRQ • 2001 - Standing USPSTF Task Force

3. Who is the USPSTF? • Experts in primary care, prevention, research methods • Government supported by AHRQ but independent • Family medicine, internal medicine, pediatrics, obstetrics/gynecology, nursing, behavioral health • Scientific support from an Evidence-Based Practice Center (EPC) • Non-member liaisons from primary care clinician associations, Federal agencies

4. Thomas G. DeWitt, MD Allen Dietrich, MD, MPH Kimberly D. Gregory, MD, MPH David Grossman, MD, MPH George Isham, MD, M.P.H. Michael LeFevre, MD, MSPH Rosanne Leipzig, MD, PhD Bernadette Melnyk, PhD, RN, CPNP/NPP Lucy N. Marion, PhD, RN Virginia A. Moyer, MD, MPH Judith K. Ockene, PhD, MSEd George F. Sawaya, MD J. Sanford Schwartz, MD, AB Timothy Wilt, MD, MPH Current Members of the USPSTF Bruce N. (Ned) Calonge, M.D., M.P.H. (Chair) Diana B. Petitti, M.D., M.P.H. (Vice Chair)

5. Task Force Activities • Provide evidence-based scientific reviews of preventive health services for use in primary healthcare delivery settings for patients without recognized signs or symptoms of target condition • Age- and risk-factor specific recommendations for routine practice • Recommendations include: • Screening tests • Counseling • Preventive medications

6. Steps in the Process for Development of Recommendations • Define questions and outcomes of interest • Define and retrieve relevant evidence • Evaluate QUALITY of individual studies • Synthesize and judge STRENGTH of available evidence • Determine balance of benefits and harms • Link recommendation to judgment about net benefits

7. Step 1: Analytic Framework on Screening for a Disease

8. Step 1: Example Analytic Framework - Prostate Cancer Prevention 1 Treat radiation, prostatectomy 3 Screen: PSA, DRE Reduced prostate cancer morbidity, mortality Early Prostate Cancer Asymptomatic Men 2 5 4 Adverse effects of screening: false positive, false negative, inconvenience, labeling Adverse effects of Rx: Impotence, incontinence, death, overtreatment

9. Steps in the Process for Development of Recommendations • Define questions and outcomes of interest • Define and retrieve relevant evidence • Evaluate QUALITY of individual studies • Synthesize and judge STRENGTH of available evidence • Determine balance of benefits and harms • Link recommendation to judgment about net benefits

10. Define and Retrieve Relevant Evidence • For each KQ developed from AF: • Create inclusion/exclusion criteria based on the key questions defined from the analytic framework • PubMed search • Cochrane search • Other database search (CINAHL, etc) • References from key articles, editorials, review articles • Expert consultation (others, TF members)

11. Steps in the Process for Development of Recommendations • Define questions and outcomes of interest • Define and retrieve relevant evidence • Evaluate quality of individual studies • Synthesize and judge STRENGTH of available evidence • Determine balance of benefits and harms • Link recommendation to judgment about net benefits

12. Assess Quality of Evidence • What do we mean by quality of evidence? “Extent to which a study’s design, conduct, and analysis has minimized selection, measurement, and confounding biases.” • Lohr, J Qual Improvement, 1999 “Extent to which one can be confident that an estimate of effect is correct” • GRADE , BMJ 2004

13. Evaluate Quality of Individual Studies • Good: • Uses a credible reference standard • Reliability of test assessed • Includes large number of subjects • Fair: • Uses reasonable although not best standard • Interprets reference std independent of screening test • Moderate sample size • Poor: Has fatal flaw such as: • Uses inappropriate reference standards • Biased ascertainment of reference standard • Very small sample size or very selected patients.

14. Steps in the Process for Development of Recommendations • Define questions and outcomes of interest • Define and retrieve relevant evidence • Evaluate quality of individual studies • Synthesize and judge STRENGTH of overall evidence • Determine balance of benefits and harms • Link recommendation to judgment about net benefits

15. Critical Appraisal Questions • Do the studies have the appropriate research design to answer the key questions? • To what extent are the existing studies high quality? • To what extent are the results of the studies generalizable (or “applicable”) to the general US primary care population and situation? • How many studies have been conducted that address the key questions? How large are the studies? • How consistent/coherent are the results of the studies? • Are there additional factors that assist us in drawing conclusions about the certainty of the evidence? (e.g., presence or absence of dose-response effects; fit within a biologic model)

16. Step 4: Synthesize and Judge Strength of Overall Evidence • Evidence reports • Evidence tables summarizing studies • Narrative discussing overall strength of evidence • Meta-analysis • Modeling • Systematic reviews from others – Cochrane, etc.

17. Step 4: Synthesize & Judge Strength of Key Question Evidence Convincing: Well-designed, well-conducted studies in representative populations that directly assess effects on health outcomes Adequate: Evidence sufficient to determine effects on health outcomes, but limited by number, quality, or consistency of studies, generalizability to routine practice, or indirect nature of the evidence. Inadequate : Insufficient evidence to determine effect on health outcomes due to limited number or power of studies, important flaws in their design or conduct, gaps in the chain of evidence, or lack of information on important health outcomes

18. Certainty • Definition: The U.S. Preventive Services Task Force defines certainty as “likelihood that the USPSTF assessment of the net benefit of a preventive service is correct”. The net benefit is defined as benefit minus harm of the preventive service as implemented in a general, primary care population. The USPSTF assigns a certainty level based on the nature of the overall evidence available to assess the net benefit of a preventive service.

19. Levels of Certainty • High: The available evidence usually includes consistent results from well-designed, well-conducted studies in representative primary care populations. These studies assess the effects of the preventive service on health This outcomes. conclusion is therefore unlikely to be strongly affected by the results of future studies. • Moderate: The available evidence is sufficient to determine the effects of the preventive service on health outcomes, but confidence in the estimate is constrained by factors such as: the number, size, or quality of individual studies; inconsistency of findings across individual studies; limited generalizability of findings to routine primary care practice; or lack of coherence in the chain of evidence. As more information becomes available, the magnitude or direction of the observed effect could change, and this change may be large enough to alter the conclusion. • Low: The available evidence is insufficient to assess effects on health outcomes. Evidence is insufficient because of: the limited number or size of studies; Important flaws in study design or methods; inconsistency of findings across individual studies gaps in the chain of evidence; findings not generalizable to routine primary care practice; or a lack of information on important health outcomes. More information may allow an estimation of effects on health outcomes.

20. Steps in USPSTF Process for Development of Recommendations • Define questions and outcomes of interest • Define and retrieve relevant evidence • Evaluate quality of individual studies • Synthesize and judge STRENGTH of overall evidence • Determine balance of benefits and harms • Link recommendation to net benefits

21. Step 5: Determine Balance of Benefits and Harms Estimate Magnitude of Net Benefit Benefits – Harms = Net Benefit

22. Step 5: Determine Balance of Benefits and Harms: Assessing Harms • Potential harms real but hard to quantify • Include psychological and physical consequences of false-positives, false-negatives, “labeling”, over treatment • Opportunity costs • Magnitude and duration of harm subjective, hard to compare to benefits • May translate into QALYs to compare • NNH

23. Step 5: Determine Balance of Benefits & Harms: Assessing Magnitude of Net Benefit • No explicit criteria for magnitude • Substantialbenefit : impact on high burden or major effect on uncommon outcome • Problems: requires evidence on harms and common metric for benefit and harms

24. Steps in the Process for Development of Recommendations • Define questions and outcomes of interest • Define and retrieve relevant evidence • Evaluate quality of individual studies • Synthesize and judge STRENGTH of overall evidence • Determine balance of benefits and harms • Link recommendation to net benefits

25. Step 6: Link recommendation to net benefits:USPSTF Grades of Recommendations

26. Step 6: Link recommendation to net benefits: Insufficient Evidence • Lack of evidence on harms or benefits • Poor quality of existing studies • Good quality studies with conflicting results

27. Communicating USPSTF Recommendations

28. USPSTF Conclusion about Evidence and Net Benefit

29. What is Certainty? • Likelihood that the USPSTF assessment of the net benefit of a preventive service is correct • Net benefit is defined as benefit minus harm of the preventive service. • USPSTF assigns a certainty level based on the nature of the overall evidence available to assess the net benefit of a preventive service.

30. Increasing use of USPSTF recommendations AHRQ develops products based on the USPSTF recommendations to engage different audiences

31. Decision Support Resources for Different Audiences • Clinicians • Consumers • Businesses, Employers, and Health Care Purchasers

32. Electronic Resources for Clinicians • ePSS – electronic Preventive Services Selector Tool • Search USPSTF recommendations by age, sex and risk factors • Available as a web-based tool or can be downloaded to your PDA • www.epss.ahrq.gov

33. Print Resources for Clinicians Publication of Recommendations and Evidence Annual Pocket Guide

34. Resources for Clinicians and Consumers • Based on recommendations of the USPSTF. • At-a-glance wall chart for appropriate preventive services based on age, sex, and risk status. • To be used in prompting shared decision-making between consumers and their primary care clinician.

35. Resourcesto inform consumer decisions • Print materials • Checklists and brochures based on the clinical preventive services recommendations of the USPSTF: • Men: Stay Healthy at Any Age – Checklist for Your Next Checkup* • Women: Stay Healthy at Any Age –Checklist for Your Next Checkup* • Pocket Guide to Good Health* - Adults, Children, and Adults 50+ • Adult Preventive Care Timeline – Wall chart *Available in English and Spanish

36. Electronic Resources for all audiences Website: www.preventiveservices.ahrq.gov • Access to • USPSTF Recommendations, Reports, and Methods Papers • Prevention Dissemination and Implementation Information and Materials • Links to our partners and their information, such as NCI’s Cancer Control P.L.A.N.E.T. and Purchaser Guide to Clinical Preventive Services.