EBM Review

1. EBM Review Escher Howard-Williams 6/13/08

2. Overview Intention is to review basic concepts of EBM (mostly stuff you know - refresher) Types of Studies Types of analyses based on study type Statistics Bias

4. Randomized Controlled Trial (RCT) A true experiment, in which the researcher randomly assigns some patients to at least one maneuver (treatment) and other patients to a placebo, or usual treatment Key feature = the classic way to evaluate effectiveness of drugs Prospective

5. RCT - ITT In an intent-to-treat analysis patients would be analyzed according to the groups for which they were originally assigned. Done to avoid the effects of crossover and dropout, which may break the randomization Intention to treat analysis provides information about the potential effects of treatment policy rather than on the potential effects of specific treatment

6. Cohort Study A longitudinal study that begins with the gathering of two groups of patients (the cohorts), one which received the exposure of interest, and one which did not, and then following this group over time (prospective) to measure the development of different outcomes (diseases) Comparison of risk (smokers vs. nonsmokers and risk of lung cancer)

7. Cohort Study Good way to determine risk relating to exposure to a harmful substance. Cannot exclude unknown confounders, blinding is difficult, and identifying a matched control group may also be difficult

8. Case-control study This type of research begins by identifying patients with the outcome (disease) of interest and looks backward (retrospective) to see if they had the exposure of interests Cases, people who have the outcome (disease) in question, are linked with controls, people from the same population without the outcome (disease) Pts with lung cancer versus those without it assessing for tobacco exposure

9. Case Control Study The classic study design for the initial investigation of cause-effect relationships Odds ratios (but not of absolute risks). A case-control study cannot be used to prove cause-effect relationships The case-control study is ideal for rare diseases or disease that takes many years to develop

10. Cross-sectional study Prevalence study. Survey of an entire population for the presence or absence of a disease and/or other variable in every member (or a representative sample) and the potential risk factors at a particular point in time or time interval Exposure and outcome are determined simultaneously Cannot establish causation, subject to bias

11. Community Trial An entire community receives a treatment or preventative measure to determine if it works in the "real world".

12. Crossover Study Design The administration of two or more experimental therapies one after the other to the same group of patients

13. Case Series A collection of anecdotes (patients with an outcome of interest). No control group is involved.

14. Case Report An anecdote. To get into the medical literature, a case report typically must convey the message "man bites dog."

15. Diagnostic Papers Terms involved: Prevalence, Incidence Sensitivity, Specificity PPV NPV LR OR

16. Prevalence Prevalence refers to the number of individuals with a given disease at a given point in time divided by the population at risk at that point in time

17. Incidence Incidence represents the number of NEW events that have occurred in a specific time interval divided by the population at risk at the beginning of the time interval. The result gives the likelihood of developing an event in that time interval

18. Sensitivity Number of patients with a disease that test positive. Based on gold standard. A test with high sensitivity will not miss many patients who have the disease (ie, few false negative results) Sensitivity: true positives/(true positives + false negatives) Not affected by prevalence

19. Specificity The number of patients with a negative test who do not have the disease A test with high specificity will infrequently identify patients as having a disease when they do not (ie, few false positive results) Specificity: true negatives/(false positives + true negatives) Not affected by prevalence

21. Receiver operating characteristic curves ROC curves allow one to identify the cut-off value that minimizes both false positives and false negatives. Plots sensitivity on the y axis and 1 - specificity on the x axis (True positive rate vs. False positive rate) Applying a variety of cutoff values to the same reference population allows one to generate the curve. For the vast majority of cases, as one moves from left to right on the ROC curve the sensitivity increases while the specificity decreases

22. Predictive values The positive predictive value of a test represents the likelihood that a patient with a positive test has the disease The negative predictive value represents the likelihood that a patient who has a negative test is free of the disease PPV and NPV are affected by the prevalence of the disease

23. PPV and NPV Calcs PPV = true positives /(true positives + false positives) NPV = true negatives /(false negatives + true negatives)

25. Likelihood ratio Likelihood that a given test result would be expected in a patient with a disease compared to the likelihood that the same result would be expected in a patient without that disease Used because predictive values are limited given their disease prevalence LR represents a measure of the odds of having a disease relative to the prior probability of the disease The estimate is independent of the disease prevalence �ONE, FIVE AND TEN� Rule > 10 or < 1/10 (0.1) = very useful 5-10 or 1/10 -1/5 = moderately useful no change (not useful

26. LR Calculation A positive likelihood ratio is calculated by dividing sensitivity by 1 minus specificity (sensitivity/(1-specificity)) A negative likelihood ratio is calculated by dividing 1 minus sensitivity by specificity ((1-sensitivity)/specificity)

27. To use LR You take the prevalence of a disease and use that as your pre-test probability You then convert the pre-test probability to odds and multiply that by the LR (+) or LR (-) depending on if you got a positive or negative test result This gives you the post-test odd which you then convert to a post-test probability. The post-test probability is also the PPV Pre-test probability -> pre-test odds -> Pre-test odds x LR1 x LR2 x LRn -> post-test odds -> Post-test probability

28. Common LRs CAGE questionnarie for EtOH Answers �yes� to: 4 LR 120, 3 LR 19, 1 LR 1.3 V/Q scan for PE High prob 18, Int prob 1.2, Low prob 0.36, nml 0.1 Ferritin in iron def anemia <15 LR 52, 35-45 LR 1.8, 45-100 LR 0.54, >100 0.08

29. Odds Ratio The odds ratio equals the odds that an individual with a specific condition has been exposed to a risk factor divided by the odds that a control has been exposed. Used in case-control, retrospective studies In this type of study, patients with a disease are identified and compared with matched controls for exposure to a risk factor.

30. Odds Ratio This design does not permit measurement of the proportion of the population who were exposed to the risk factor and then developed or did not develop the disease; thus, the relative risk or the incidence of disease cannot be calculated However, in case-control studies, the odds ratio provides a reasonable estimate of the relative risks

31. Therapy Papers Terms Involved: EER CER RR RRR ARR NNT

33. Event Rate Experimental Event Rate (EER) Event rate in treated group a/n1 or a/(a+b) Control Event Rate (CER) Event rate in control group c/n0 or c/(c+d)

34. Absolute risk Risk of having a disease If the incidence of a disease is 1 in 1000, then the absolute risk is 1 in 1000 or 0.1%.

35. Relative risk Event rate in treatment group divided by the event rate in the control group. RR (aka Risk Ratio) is used in randomized trials and cohort studies. RR = EER/CER RR can be calculated from studies in which the proportion of patients exposed and unexposed to a risk is known. ie a cohort study, in which a group of patients who have variable exposure to a risk factor of interest are followed over time for an outcome

37. Absolute Risk Reduction ARR = CER-EER Difference in the event rate between the control group (CER) and treatment group (EER). It reflects the additional incidence of disease related to an exposure taking into account the background rate of the disease.

38. Relative Risk Reduction RRR=CER -EER / CER Percent reduction in events from treated group compared to control group

39. Number Needed to Treat NNT is the reciprocal of the absolute risk reduction The number of patients who need to be treated to prevent one bad outcome

40. Note on Harm In harm papers similar calculations apply Absolute Risk Increase (ARI) Relative Risk Increase (RRI) Number Needed to Harm (NNH)

41. Reliability Reliability refers to the extent to which repeated measurements of a relatively stable phenomenon fall closely to each other. Several different types of reliability can be measured. Examples include inter- and intraobserver reliability and test-retest reliability.

42. Validity Validity refers to the extent to which an observation reflects the "truth" of the phenomenon being measured. Several types can be measured such as content (the extent to which the measure reflects the dimensions of a particular problem) construct (the extent to which a measure is affirmed by an external established indicator) criterion validity (the extent to which a measure can predict an observable phenomenon). These types of validity are often applied to questionnaires

43. Internal and external validity Internal validity addresses the question of whether the results of clinical research are correct for the patients in the study and is threatened by bias and chance External validity addresses the question of whether the results of the research apply to patients outside of the study population

44. P value Measured probability of a finding occurring by chance alone given that the null hypothesis is actually true

45. Confidence interval A point estimate from a sample population may not reflect the "true" value from the entire population Confidence intervals give an idea of how likely the sample mean represents the population mean. The calculation of a confidence interval considers the standard deviation of the data and the number of observations. Thus, a confidence interval narrows as the number of observations increases, or its variance (dispersion) decreases.

46. Errors Two potential errors are commonly recognized when testing a hypothesis: A type I error (aka alpha) is the probability of incorrectly concluding that there is a statistically significant difference in a dataset. Alpha is the number after a p-value A type II error (aka beta) is the probability of incorrectly concluding that there was no statistically significant difference in a dataset. Often reflects insufficient power of the study

48. Power The probability of detecting an effect in the treatment vs. control group if a difference actually exists. Typical power probabilities are 80% or greater. Power = 1 - beta

49. Power Calculated as 1 - beta,refers to the ability of a study to detect a true difference A "power calculation" may be performed prior to conducting a study to ensure that there are a sufficient number of observations to detect a desired degree of difference The larger the difference, the fewer the number of observations that will be required

50. Level of Significance The probability of incorrectly rejecting the null hypothesis There is a difference between two groups when actually there is none. Otherwise known as the probability of Type I error. Frequently p value of 0.01 or 0.05

51. Bias Selection Bias: The sample population chosen is not representative of the population at risk. An appropriate spectrum of patients were not included in the study. Measurement Bias: Being studied can affect outcome. Also how you measure can affect outcome. Confounding Bias: Occurs when two factors are closely associated and the effects of one confuses or distorts the effects of the other factor on the outcome. The distorting factor is a confounding variable.

52. Bias Recall Bias: The recall of exposures or events may differ in cases and controls. Questions may be asked more times and more intensively in cases compared to controls. Referral Bias: Physicians and medical centers may attract individuals with specific disorders or exposures. Volunteer Bias: Volunteers may exhibit exposures or outcomes which may differ from nonvolunteers

53. Bias Withdrawal Bias: Patients who withdraw from studies may differ systematically from those who remain. Attention Bias: When subjects systematically alter their behavior when they are being observed. Therapeutic Personality Bias: Occurs when the observer is not blinded. The observer's beliefs about therapeutic effectiveness may influence outcomes and their measurements.

54. Bias Investigator Bias: Occurs when the interviewer is aware (not blinded) of the outcome variable. An unblinded interviewer may be more vigorous in searching for the exposure of interest. Gold Standard Review Bias: A form of investigator bias that occurs when the investigator knows the results of the gold standard test when the new diagnostic test in interpreted.

55. Bias Index Test Review Bias: A form of investigator bias that occurs when the investigator knows the results of the new diagnostic test when the gold standard test in interpreted Verification Bias: Occurs when patients with negative test results are not evaluated with the gold standard test

56. References Sackett, DL, Straus, SE, Richardson, WS, et al. Evidence-based medicine. How to practice and teach EBM, 2nd edition, Churchill Livingstone, Edinburgh 2000. Fletcher, RH, Fletcher, SW, Wagner, EH. Clinical Epidemiology: The Essentials, 2nd ed, Williams & Wilkins, Baltimore 1988. Up to date http://www.musc.edu/dc/icrebm/index.html Resource from MUSC - EBM overview