Diagnostic Test Studies

1. Diagnostic Test Studies Assessing Validity Understanding Results

2. Questions about Validity • Was there an independent, blind comparison with a reference standard? • Did the patient sample include an appropriate spectrum of patients to whom the diagnostic test will be applied in clinical practice?

3. Independent, blind comparison with a reference standard? • Reference standard: the defining test for the condition (“gold standard”) or the best available test. • Independent: Everyone gets both the new test and the reference standard • Blind: The person scoring the new test doesn’t know the reference diagnosis, and vice versa

4. Appropriate patient spectrum? • Characteristics of the patients who are studied should be similar to the patients in whom you want to apply the test • Bad: studying a test in patients in an advanced state of disease that you mean to apply to patients in an early stage • Okay: differences in prevalence of the condition

5. Typical study design • The typically best study design is a prospective cohort study: • Identify cohort of appropriate spectrum • Given each patient new test and reference standard in a blind fashion • Compare test results

6. Working with results • When a patient presents, the clinician thinks they have some (“prior” or “pre-test”) probability of being sick • Based on clinician suspicions • Based on prevalence of illness in population • The purpose of a diagnostic test is to change that probability to a post-test probability that is high enough to act (or low enough to ignore) • A positive test should increase the probability • A negative test should decrease the probability • Better tests lead to bigger changes in probabilities.

7. Getting it right and wrong • Given that you know the patient’s true disease state (from the reference standard) • A test can be right in two ways • True positive (test says a sick person is sick) • True negative (test says a well person is well) • A test can go wrong in two ways • False positive (test says a well person is sick) • False negative (test says a sick person is well)

8. The 2x2 table

9. Example of a 2x2 table Study prevalence: 100/1000 (10%) of those studied were sick.

10. Sensitivity: How good is the test when you’re sick? This test correctly picks up 95/100 people who are sick. Its sensitivity is 95%

11. Specificity: How good is the test when you’re healthy? This test correctly classifies 100/900 people who are healthy. Its specificity is 89%

12. Using sensitivity/specificity • Sensitivity and specificity are test characteristics that are independent of disease prevalence. • With sensitivity, specificity, and your pre-test probability, you can compute your patient’s post-test probability if they have a positive or negative test.

13. Using sensitivity/specificity • A negative result on a highly sensitive test rules out the disease, because if you were sick, a highly sensitive test would be positive. Mnemonic: SnNout • A positive result on a highly specific test rules in the disease, because if you were well, a highly specific test would be negative. Mnemonic: SpPin

14. Likelihood ratios • A likelihood ratio is the probability that, for a given test result, the patient is in the sick rather than the well population. • Each test result (positive, negative) has a likelihood ratio (LR+, LR-) • LR+ should be greater than 1 • LR- should be less than 1 (fractional) • LR of 1 means the test result adds no new information (result is equally likely to occur in a sick as in a well person)

15. Calculating LRs • LR+ = sensitivity / 1 – specificity • LR- = 1 – sensitivity / specificity • But life’s too short, so let a computer or calculator do it…

17. The nomogram