Designing Clinical Research Lecture #4 Causal Inference – or Truth in the Universe

1. Designing Clinical ResearchLecture #4Causal Inference – orTruth in the Universe Deborah Grady, MD, MPH

2. FRAMEWORK • Untruth – spurious associations • Chance (small sample size) • Bias (selection and other biases) • Truth – real associations, not always causal • Effect – cause • Effect – effect (confounding) • Cause – effect (truth in the universe!)

3. RANDOMIZED CLINICAL TRIALS • Eliminate effect-cause • Best design to minimize confounding • Major pitfalls • Low power • Not randomized • Unblinded • Incomplete follow-up

4. Case study: estrogen and chd in women

5. ESTROGEN AND CHD IN WOMEN • RQ: Does estrogen therapy reduce CHD risk in postmenopausal women? • Design:Cross-sectional • Subjects:20 postmenopausal women – entire population of my Tuesday clinic • Measurements:estrogen therapy (ever/never) self-report; CHD (yes/no) chart review

6. ESTROGEN AND CHD IN WOMEN CROSS-SECTIONAL STUDY RR = 0.5 95% CI 0.08-3.2; p = 0.4

7. ESTROGEN AND CHD IN WOMEN • RQ: Does estrogen therapy reduce CHD risk? • Design: Case-control • Subjects: 1000 women admitted to SFGH over 5years with discharge diagnosis of CHD (ICD-9 codes) and1000 women identified by random digit dialing in SF who report no CHD • Measurements: CHD based on discharge diagnosis; estrogen therapy based on self-report

8. ESTROGEN AND CHD IN WOMEN CASE-CONTROL STUDY

9. ESTROGEN AND CHD IN WOMEN • RQ: Does estrogen therapy reduce CHD risk?? • Design: Case-control • Subjects: 1000 women admitted to Kaiser over 5years with discharge diagnosis of CHD and 1000 women admitted to Kaiser over the same period with no discharge diagnosis of CHD • Measurements: CHD based on discharge diagnosis; estrogen therapy based on computerized pharmacy records

10. ESTROGEN AND CHD IN WOMEN CASE-CONTROL STUDY

11. CONFOUNDING

12. INTERACTION

13. CONTROLLING CONFOUNDING • Design stage • Matching • Specification • Randomization • Analysis stage • Stratification • Multivariate modeling

14. ESTROGEN AND CHD IN WOMEN • RQ: Does estrogen therapy reduce CHD risk?? • Design: Prospective cohort • Subjects: 59,337 PM nurses followed for 16 years • Measurements: Self-reported estrogen use; self-reported CHD events validated by chart review • Analysis: Multivariate logistic regression – age, ethnicity, education, blood pressure, diabetes, smoking, alcohol, family history of CHD and hypercholesterolemia

15. NURSES’ HEALTH STUDY Grodstein, NEJM, 1996

16. RISK FOR CORONARY HEART DISEASE IN ESTOGEN USERS VS. NONUSERS CohortStudies • Grodstein, 2000 • Falkeborn, 1992 • Wolf, 1991 • Henderson, 1991 • Sullivan, 1990 • Avila, 1990 • Criqui, 1988 • Petitti, 1987 • Bush, 1987 • Wilson, 1985 Angiographic Studies • McFarland, 1989 • Sullivan, 1988 • Gruchow, 1988 Case-Control Studies • Mann, 1994 • Rosenberg, 1993 • Croft, 1989 • Beard, 1989 • Szklo, 1984 • Ross, 1981 • Bain, 1981 • Adam, 1981 • Rosenberg, 1980 • Pfeffer, 1978 • Talbott, 1977 • Rosenberg, 1976 RR = 0.65 Summary Relative Risk s 0.01 0.1 1 10 Relative Risk

17. POTENTIAL MECHANISMS FOR CHD BENEFIT OF ESTROGEN THERAPY • Improves lipoproteins • Reduces LDL 10-15% • Increases HDL 10-15% • Retards atherosclerosis • Prevents coronary vasoconstriction

18. ESTROGEN AND CHD IN WOMEN Observational findings • Strong association • Consistent association • Plausible biologic mechanism CAUSALITY

19. REASONS TO BE CAUTIOUS • Observational findings are susceptible to bias and confounding • Estrogen has known risks • Estrogen was a preventive therapy widely used among healthy women

20. 4 IMPORTANT FEATURES OF RCTs

21. POWER OF THE PLACEBO Arthroscopic debridement of the knee • In unblinded trials • Reduced knee pain about 60% • In blinded trials • Reduced knee pain about 60% • Subjects who underwent debridement • Subjects who underwent sham debridement Mosely et al, NEJM, 2002

22. BLINDING TO AVOID DIFFERENTIAL OUTCOME ADJUDICATION Canadian Cooperative MS Trial • 165 patients with multiple sclerosis • Plasma exchange + cyclophosphamide + prednisone • Sham plasma exchange + placebo meds • Outcome = structured neurologic exam by blinded and unblinded neurologists • More improvement with plasma exchange by unblinded, but not blinded assessment Noseworthy, Neurology, 1994

23. BLINDING TO AVOID CO-INTERVENTION • Unintended effective interventions • Participants use other therapy or change behavior • Study staff, medical providers, family or friends treat participants differently • Nondifferential decreases power • Differential causes bias

24. ESTROGEN AND CHD IN WOMEN RQ: Does estrogen therapy reduce CHD risk? Design: Randomized trial Subjects: 2500 PM women with CHD Intervention: Estrogen + progestin vs. placebo Measurements: Predictor = treatment; Outcome = CHD death or nonfatal myocardial infarction

25. ESTROGEN AND CHD IN WOMEN RANDOMIZED TRIAL Where are the other 750 women randomized to HT?

26. HEART AND ESTROGEN-PROGESTIN REPLACEMENT STUDY (HERS) • 2763 postmenopausal women <80 yo with documented CHD and a uterus • Randomized to estrogen plus progestin or identical placebo • Followed every 4 months for 4.2 years • Separate gynecology group managed bleeding • Outcome = nonfatal MI and CHD death

27. HERS TRIAL PROFILE

28. HERS: BASELINE CHARACTERISTICS

29. Cumulative % Years CHD EVENTS IN HERS Hulley, Grady, JAMA 1998

30. HERS: PRIMARY OUTCOMES

31. HERS: CARDIOVASCULAR OUTCOMES

32. HERS vs. OBSERVATIONAL STUDIES Why did the findings of HERS differ? • HERS design different • Adverse effect of added progestin • No benefit in women with CHD • Observational findings wrong • Selection bias – comparison groups differ • Adherence bias

33. WOMEN’S HEALTH INITIATIVE • 2 NIH-funded concurrent randomized trials in postmenopausal women without CHD • Uterus – E+P vs. placebo (16,606) • No uterus – estrogen vs. placebo (10,739) • Multiple outcomes • Planned follow-up 9 years • Both trials stopped early due to harm or lack of benefit

34. WHI RESULTS

35. RANDOMIZED TRIAL FINDINGS • No reduction in risk for CHD with estrogen or E+P in women with or without CHD • Why did observational studies find benefit? • Populations studied different • Younger women in observational studies • Observational studies exclude “early users” • Confounding: hormone users in observational studies were healthier than nonusers in unmeasureable ways

36. BENEFIT OF ADHERENCE TO MEDICATION

37. ARE OBSERVATIONAL STUDIES USELESS? • NO • Generate important hypotheses • Provide only answer if trial not feasible • Generally produce correct answer • But bias and confounding always an issue • Particularly problematic for interventions that require selection and adherence

38. SUMMARY • Untruth – spurious associations • Chance (small sample size) • Bias (selection and other biases) • Truth – real associations, not always causal • Effect – cause • Effect – effect (confounding) • Cause – effect (truth in the universe!)