EBM-Diagnostic Testing K. Mae Hla, MD, MHS Primary Care Faculty Development Fellowship

1. EBM-Diagnostic Testing K. Mae Hla, MD, MHS Primary Care Faculty Development Fellowship November 13, 2010

2. Objectives • Develop pre-test probabilities • Derive treatment thresholds • Appraise evidence about a diagnostic test -validity, accuracy and applicability • Calculate the results of diagnostic tests -sensitivity, specificity and likelihood ratios • Apply evidence to patient care decisions

3. The Diagnostic Process • Working diagnosis- pretest probability • With each new finding/test we move from the pre-test probability to a new post-test probability • Clinicians estimate probability of disease using probabilistic, prognostic and pragmatic approaches • Compare disease probabilities to two thresholds

5. Applying Diagnostic Tests Example #1 8-year-old with fever, sore throat, swollen cervical glands and tonsillar exudates. No h/o cough. You order a rapid strep test.

6. What’s your pretest probability of the patient having group A strep pharyngitis? How much of a change would help you decide to treat, not treat or test further?

7. 0% 100% Probability of Strep Pharyngitis Treatment Thresholds Tx No Tx ZONE OF UNCERTAINTY 5% 90%

8. Rapid Strep Test Results Rapid Strep test result comes back negative How does the rapid strep test result change the probability of the patient having or not having the disease? A positive rapid strep test raises post test probability of strep pharyngitis to 85% in one study A negative strep test decreases probability to 12 %

9. Pre-test Prob = 40% LR+ = 7.2 LR- = 0.24

10. Treatment Thresholds ZONE OF UNCERTAINTY 0% 12% 100% Probability of strep when rapid strep test is negative Tx No Tx X 5% 90%

11. Example #2 18-year-old female with ankle pain after a roller-blading accident. States unable to walk on her injured ankle. Exam demonstrates a slightly swollen ankle but no tenderness noted. Able to bear weight and take 4 full steps upon encouragement. What is the probability of ankle fracture? Do you need to order an ankle x-ray?

12. Ottowa Ankle rule An ankle x-ray is only necessary if there is pain near the malleoli and any of the following findings are present: inability to bear wt. both immediately and in the ED bone tenderness at the posterior edge or tip of the malleolus

14. How accurate is the Ottowa ankle rule in ruling out ankle fracture? A prospective validation study in > 1000 pts presenting to the ED with ankle pain Likelihood ratio + = 1.96 Likelihood ratio - = 0

15. Pre-test Prob = 10% LR+ = 1.96 LR- = 0

16. Case 1 75-year-old woman with a hemoglobin of 10, MCV was 80 on routine checkup, a negative history and physical except osteoarthritis, and on no meds likely to suppress her marrow or cause a bleed Her probability of iron deficiency was 50% You want to avoid doing a bone marrow and order serum ferritin to diagnose iron deficiency anemia

17. Case 1 P: In an elderly symptomless woman with mild anemia I: how useful is serum ferritin C: O: in diagnosing iron deficiency anemia T(ype of question): Diagnosis T(ype of study): Prospective Cohort *Diagnosis of Iron Deficiency Anemia in the Elderly (Guyatt, et al. Am J Med, 1990;(88):205-209

18. Three Main Questions Validity-Is this evidence about the accuracy of a diagnostic test valid? Results-Does this evidence show that this test can adequately distinguish patients who do and do not have the disorder? Applicability-How can I apply this valid, accurate diagnostic test to a specific patient?

19. Validity • Measurement: was the gold standard measured independently? • Representative: was the test evaluated in appropriate spectrum of patients? • Ascertainment: was the reference test ascertained regardless of the diagnostic test?

20. Validity- Measurement • All patients in study should have both the diagnostic test in question (blood test, history, physical exam) and the gold/reference standard test (autopsy, bone marrow, biopsy, angiogram) • Independent- test not part of gold standard, decision to perform gold standard should not depend on result of diagnostic test under study • Blinding- reference test readers should be unaware of results to avoid bias if tests/gold standard have subjective component- x-rays, biopsy, slides

21. Validity: Measurement • Was there an independent blind comparison with a reference gold standard? • The gold standard was the bone marrow aspirate results • All patients got the serum ferritin and bone marrow done independently • Marrow aspirates and iron deficiency status was determined by 2 hematologists unaware of the lab result

22. Validity: Representative • Was the diagnostic test evaluated in an appropriate spectrum of patients? • Examples: risk markers such as CEA were initially done in high risk patients

23. Validity: Representative • Diagnostic uncertainty • Patients with mild as well as severe symptoms • Patients with early as well as late disease • Patients with other commonly confused diagnoses

24. Study spectrum representative? • Consecutive patients age 65 or older with anemia were recruited • 36% of patients had iron deficiency anemia • 44% had anemia of chronic disease • 8% megaloblastic anemia • Patients with other commonly confused disorders- different types of anemia and chronic medical conditions were included

25. Validity: Ascertainment • Was the reference standard ascertained regardless of the diagnostic test result? • Did all patients in the study both with and without iron deficiency anemia get the bone marrow done? • Yes

26. Ascertainment-Continued • Patients with negative diagnostic test may not get the gold standard done if the latter is invasive • How do we prove for sure that the ones with negative tests truly do not have the disease or vice versa? • Other ways to establish reference test

27. In the Pioped study looking at the utility of V/Q scan in patients with suspected pulmonary embolism-all patients with negative V/Q scan did not get pulmonary angiogram • Clinical followup in a year was the additional reference standard to not miss patients with false negative VQ results

28. Results • Does the test accurately distinguish between patients with and without the disorder? • Sensitivity and specificity • Likelihood ratios

33. Sensitive test-rules out the disease (SnNout) • Test with high sensitivity (high TPR and very low false negative rate), negative test rules out the disease • Examples: • loss of retinal vein pulsation in increased intracranial pressure-the presence of pulsation (negative test) rules out IIP • HIV antibody- negative test rules out HIV

34. Specific test – rules in the disease (SpPIN) • Test with high specificity (high TNR, very low FPR)-positive test rules in the diagnosis • Features of child with Down’s syndrome-very specific • Presence of features (positive test) rules in the diagnosis • Western blot confirmatory testing for HIV- high specificity: positive test rules in HIV disease

35. likelihood of the test result in patients with the disease likelihood of the same result in patients without disease Likelihood Ratio LR =

36. (+)LR= + test result in pts with dz (-)LR= - test result in pts with dz + test result in pts without dz - test result in pts without dz

37. (+)LR= + test result in pts with dz (-)LR= - test result in pts with dz + test result in pts without dz - test result in pts without dz = Sn/1-Sp = 1-Sn/Sp

38. probability of the test result in patients with the disease probability of the same result in patients without disease Likelihood Ratio LR =

39. Likelihood Ratio Pre-Test Probability Pre-Test Odds Post-Test Odds Post-Test Probability Odds = Probability/1-probability Probability = Odds/1 + Odds

41. What do all these numbers mean?!? • L.R.s indicate by how much a given diagnostic test result will raise or lower the pre-test probability of the target disorder • L.R. of 1 = post-test probability is same as pre-test probability • L.R. > 1 increases the probability that the target disorder is present; the higher the L.R., the greater the increase • L.R. < 1 decreases the probability of the target disorder; the smaller the L.R., the greater the decrease

42. Effects of different likelihood ratios • >10 or <0.1 generate large and often conclusive changes from pre- to post-test probability • 5-10 and 0.1-0.2 generate moderate shifts in pre- to post-test probability • Depending on pre-test probability, change may or may not be large enough to influence Rx decision

43. Back to our patient • Our patient’s serum ferritin comes back at 40 mmol/L • How should we put all this together?

45. Low ferritin (<45) in diagnosing Fe def anemia Prevalence (study pre-test probability) = 85/235= 36% Sensitivity = True positive / all with disease = a/a+c = 70/85 = 82% Specificity = True neg / all without disease = d/b+d = 135/150 = 90%

46. Low ferritin (<45) in diagnosing iron deficiency anemia L.R.+ = sensitivity/(1-specificity) = 82%/10% = 8.2 L.R. - = (1-sens)/spec = 18%/90% = 0.2

47. Simplifying Likelihood Ratio Calculations

48. Calculating Likelihood Ratios at 45 cut point

49. Pre-test Prob = 36% LR+ = 8.2 LR- = 0.2

50. Applying the Test to the Patient • Is the diagnostic test available, affordable, accurate and precise in our setting? • Yes