Outcomes in Clinical Trials

1. Outcomes in Clinical Trials Rick Chappell, Ph.D. Professor, Department of Biostatistics and Medical Informatics University of Wisconsin School of Medicine & Public Health chappell@stat.wisc.edu42 – Week 1, Lecture 1 BMI 542 – Week 1, Lecture 2 (some slides adapted from D. DeMets’)

2. “What’s the Question?” Dr. Max Halperin NHI (NHLBI), National Institutes of Health

3. 1. Primary vs. Secondary Question • Primary • most important, central question • feasible! • ideally, only one • stated in advance • basis for design and sample size • Secondary • related to primary • stated in advance • limited in number

4. Examples (1) • Physicians Health Study (PHS) • aspirin vs placebo • primary: total mortality • secondary: fatal + nonfatal myocardial infarction (MI)

5. Examples (2) • Eastern Cooperative Oncology Group (ECOG - 1178) • tamoxifen vs placebo • primary: tumor recurrence/relapse, disease-free survival (does this include mortality?) • secondary: total mortality

6. Examples (3) • Chronic Study of Intermittent Positive Pressure Breathing (IPPB) • long-term intermittent positive pressure breathing vs. nebulizer • primary: forced expiratory volume (FEV1) • secondary: quality of life

7. Examples (4) • ON-Q for pain in Gyn.-Onc. Surgery (Kushner, et al., Obs&Gyn. 106, 2005) • Intra-operative bupivacaine vi catheter in hysterectomies (etc.) • Primary: pain score • Secondary: amount of analgesic used • Secondary: number of days in hospital (see accompanying editorial by Collins)

8. Examples (5) • Vitamin K for bone density (Binkley, et al., J Bone Min Res 24, 2009) • Vitamins K + D vs. D alone • Primary: (reduce) undercarboxylated osteocalcin • Secondary: Bone density spine • Secondary: Bone density femur • Secondary: Bone geometry

9. Examples (6) • Australian Falls Study • Mortality? • Mortality from falls? • Bone fracture? • Fall-related Injury? • Treatment (hospitalization) for fall-related injury? • Number of falls or days with falls? See patient “diary”.

10. 2. Subgroup Questions • Questions about effect of therapy in a sub-population of subjects entered into the trial • Example: Does lowering blood pressure have equal benefit for: 1. Males & females 2. White and non-white 3. Mild and severe hypertension

11. Subgroup Considerations • Rules for Subgroups 1. Stated in advance (in protocol) 2. Limited in number 3. Interpreted cautiously, qualitatively 4. Look for consistency of results • May be used to 1. Confirm or answer specific questions generated in a previous trial (e.g. Metroprolol <65 vs. >65 age total mortality 2. Generate new hypothesis to be tested in some future trial 3. Check consistency of primary outcomes

12. Example - Subgroup Concern • Second International Study of Infarct Survival (ISIS 2) • 2 x 2 factorial design (aspirin vs. placebo and streptokinase vs. placebo) • vascular and total mortality in patients with an acute myocardial infarction (MI) • Gemini or Libra astrological birth signs did somewhat worse on aspirin while all other signs and overall results impressive and highly significant benefit from aspirin (Sleight, P (2000). “Debate: Subgroup analyses in clinical trials: fun to look at - but don't believe them!” Curr Control Trials Cardovasc Med 1, 25-27.)

13. Subgroup Analyses Examples: Breast Cancer: Does the benefit of treatment depend on: menopausal status, stage of disease, age, etc. AIDS: Does the benefit of treatment depend on: gender, age, initial CD4 counts, race, etc. Analyses of a trial by subgroup results in a separate statistical test for each subgroup. As a result the probability of false positive conclusions arising in the analysis of a trial will increase.

14. Three Views: • Ignore subgroups and analyze only by treatment groups. • Plan for subgroup analyses in advance. Do not “mine” data. • Do subgroup analyses but view all results with caution – see story of ISIS effect & astrological sign in Slight, “Debate: Subgroup analysis in clinical trials – fun to look at, but don’t believe them!” Curr Contr Clin Tr Cardiovas Med 2000, 1: 25-27. “: “In retrospect, perhaps one of the most important results in the ISIS trials was the analysis of the results by astrological star sign (overall p-value for benefit of aspirin < .00001, but slightly harmful for Gemini & Libra).

15. False Positive Rates The greater the number of independent subgroups analyzed separately, the larger the probability of making false positive conclusions. No. of Subgroups False Positive Rate 1 .05 2 .08 3 .11 4 .13 10 .19

16. 3. Adverse Effects • Any intervention should do more benefit than harm • Not always easy to specify in advance - many variables will be measured (clinical, laboratory) • Usually not willing or interested in demonstrating an intervention to be harmful • May be known adverse effects from earlier trials

17. What’s the Question? 4. “Natural History” • Question not related to intervention • Control group, often a “placebo,” may be used to describe how prognostic factors relate to eventual subject outcome (predictive, not causative) e.g. Coronary Drug Project: Aided greatly in defining natural history of patients following a heart attack 5. Ancillary • Questions not related at all but still of scientific interest • Usually piggy-backed onto trial • Must not interfere with trial!

18. What’s the Question? 6. Exploratory • Most studies conducted to test some hypothesis • Most studies can generate new hypotheses • Multiple analyses often conducted • increased false positive (Type I) error rate • Could demand reduced significance level (or p-value) for each test • e.g. /K (Bonferroni adjustment assuming independent variables) •  = .05, K = 10  /K =.005 • But can’t afford this usually • Could be selective in number of primary hypotheses • Should state key comparison(s) in advance • Relegate other comparisons to either • Confirmatory or Exploratory

19. What’s the Response Variable (“endpoint”, “outcome”)? • Used to answer primary/secondary questions • Characteristics for primary/secondary outcomes 1. Well defined & stable 2. Ascertained in all subjects 3. Unbiased 4. Reproducible 5. Specificity to question

20. Response Variable • Examples 1. Falls Studies - Mortality? - Hospitalization? - Broken bone(s)? - Days (weeks, months) with falls? - Total number of falls? 2. NOTT Quality of Life? - POMS (Profile of Mood) - SIP (Sickness Impact Profile) - Pulmonary Function - Survival

21. Response Variable 3. Cardiovascular Disease Trials - Total mortality - CHD mortality - Non-fatal MI - PVC’s 4. Diabetes - Mortality - Blindness - Visual impairment - Retinopathy - Microaneurisms

22. Surrogate Response Variables • Used as alternative to desired or ideal response • Examples • Suppression of arrhythmia (sudden death) • Viral Load or T4 cell counts (AIDS or ARC) • Systolic blood pressure? • Glycosylated hemoglobin? • Used often - therapeutic exploratory (Phase I, Phase II) • Use with caution - therapeutic confirmatory (Phase III)

23. Surrogate Response Variables (2) • Frequent Criticism of Clinical Trials • Too long • Too large • Too expensive • Advantages • Perhaps smaller sample size • Detect earlier effect  shorter trial • Easier

24. Examples of FDA Approval of Drugs Using Surrogates (1) • Lower cholesterol without evidence of survival benefit • Lower blood pressure without evidence of benefit for stroke, MI, congestive heart failure, or survival • Increase bone density without evidence of decreased fractures in osteoporosis

25. Examples of FDA Approval of Drugs Using Surrogates (2) • Increase cardiac function in congestive heart failure without evidence of survival benefit • Decrease rate of arrhythmias (VPBs) without evidence of survival benefit • Lower blood glucose and glycosylated hemoglobin without evidence about diabetic complications or survival benefit

26. Surrogate Response Variables (3) • Requirements (Prentice, 1989) T = True clinical endpoint S = Surrogate Z = Treatment • H0: P(T|Z) = P(T)  P(S|Z) = P(S) • Sufficient Conditions (Conditionality = “|”) 1. S is informative about T P(T|S)  P(T) 2. S fully captures effect of Z on T P(T|S,Z) = P(T|S)

27. Concerns About Surrogates 1. Relationship between surrogate and true endpoint may not be causal, but coincidental to a third factor (confounding) 2. Other unfavorable effects of the drug 3. Effect of surrogate may correlate with one clinical endpoint, but not others

28. Time Intervention True Clinical Outcome Surrogate Disease End Point The setting that provides the greatest potential for the surrogate endpoint to be valid. Reprinted from Ann Intern Med 1996; 125:605-13. Figure 2. Reasons for failure of surrogate end points. A. The surrogate is not in the causal pathway of the disease process. B. Of several causal pathways of disease, the intervention affects only the pathway mediated through the surrogate. C. The surrogate is not in the pathway of the intervention’s effect or is insensitive to its effect. D. The intervention has mechanisms for action independent of the disease process. Dotted lines = mechanisms of action that might exist.

29. Time Reasons for failure of surrogate end points. A. The surrogate is not in the causal pathway of the disease process. B. Of several causal pathways of disease, the intervention affects only the pathway mediated through the surrogate. C. The surrogate is not in the pathway of the intervention’s effect or is insensitive to its effect. D. The intervention has mechanisms for action independent of the disease process. Dotted lines = mechanisms of action that might exist. Surrogate True Clinical Outcome A End Point Disease Intervention B True Clinical Outcome Surrogate Disease End Point Intervention C True Clinical Outcome Disease Surrogate End Point Intervention D True Clinical Outcome Surrogate End Point Disease

30. Cardiac Arrhythmia Suppression Trial Hypothesis Does suppression of arrhythmia following an MI reduce incidence of: 1. Sudden death 2. Total mortality

31. Cardiac Arrhythmia Suppression Trial Design • Randomized Double Blind • Three Drug Arms vs. Placebo • Multicenter Study • Group Sequential Data Monitoring • One Sided (0.025 Type I Error) for Benefit • Advisory One Sided (0.025) for Harm • Run-in Period (Arrhythmia Suppression)

32. Cardiac Arrhythmia Suppression Trial Early Termination in Two Drug Arms Drugs Placebo Sudden Death 33 9 Total Mortality 56 22

33. AIDS Clinical Trials • Clinical Outcomes • Death • Progression to AIDS • Progression to ARC • Surrogate Outcome • CD4 Cell Count

34. State-of-the-Art Conference • Results • AIDS/Death • *8 trials positive • 7/8 had positive CD4 cell changes • *8 trials negative • 6/8 had positive CD4 cell change • Death • *4 trials positive • 2/4 CD4 positive • *7 trials negative • 6/7 CD4 cell positive

35. Osteoporosis(Riggs et al. NEJM, 1990) • Bone loss in postmenopausal women leads to increase risk of fracture • Sodium Fluoride stimulates bone formation and increased bone mass (double) • Hypothesis • Will Fluoride treatment decrease rate of vertebral fractures? • Design • Randomized, double blind, placebo-controlled • 202 postmenopausal women randomized • All received calcium supplementation

36. Osteoporosis Fluoride Trial Results • Fluoride increased bone density by 35% • 35% (p = 0.0001) in spine • 12% (p = 0.0001) in femoral neck • Fluoride decrease bone density by 4% in wrist (p = 0.02) • Vertebral fractures higher on Fluoride (F 163, P 136, p < 0.05) • Non-vertebral fractures higher on Fluoride (72 vs. 24; p = 0.01) • Fluoride concluded not effective as a treatment for post-menopausal osteoporosis

37. Concluding Remarks on Surrogates • Surrogates play an important role in the development of Phase I, II, and pilot Phase III studies • Treatments may affect more than one mechanism • “Surrogates” do not reliably predict treatment on clinical outcome • Continued success in a given field is not even guaranteed • Reliance on “surrogates” should be minimized