Design & Interpretation of Randomized Trials: A Clinician’s Perspective

1. Design & Interpretation of Randomized Trials:A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK

2. Common problems of RCTs • Originality • Hypothesis • Allocation concealment & randomization • Evaluation of baseline data • “Intention-to-treat” analysis • Subgroup analysis

3. Is the study original? • Ground breaking research? • Does this work add to the literature in any way? • Bigger, longer? • More rigorous methodology? • Results add to a meta-analysis of previous studies? • Different population (age, sex, ethnic groups)?

4. Hypothesis & End point Many RCTs did not explicitly state their study hypotheses “The aim of this study was to compare the efficacy of a new treatment with the standard treatment…” Hypothesis 1: Treatment A is superior to the standard treatment Hypothesis 2: Treatment A is equivalent to the standard treatment

5. Hypothesis? Sample size estimation None!

6. Failure to detect a difference = Equivalence?

7. Superiority Trial The new treatment (µN) is superior to the standard treatment (µS) if the differenceexceeds by a clinically important amount (). Test hypothesis (H): µN -µS> 

8. Equivalence trial The new treatment is equivalent to the standard treatment if the maximal allowable difference does not exceed by a clinically important amount.

9. New agent is not inferior to the standard Not equivalent Favors standard treatment Not equivalent Favors new treatment Equivalence trial Equivalent - 0 + Difference

10. Assume non-inferiority if the lower limit of 95% CI is less than –5%, N=904 per group!

11. Allocation concealment & randomization • Concealment of allocation (investigators and patients not knowing the assigned treatment before randomization) • Was treatment assigned by an independent staff? • What was the method of allocation concealment? • contact with central office • blinded packages • sealed (opaque) envelopes

12. Allocation concealment

13. Comparison of baseline data Does P>0.05 indicate comparability of treatment groups? Chan et al. Lancet 1997

14. Baseline data Effect of azathioprine on the survival of patients with primary biliary cirrhosis Christensen et al. Gastro 1985

15. Baseline data Effect of azathioprine on the survival of patients with primary biliary cirrhosis Christensen et al. Gastro 1985

16. Adjusted for bilirubin P=0.01 UnadjustedP=0.10

17. P=0.04 P=0.02 Columbus Investigators. NEJM 1997

18. Comparison of baseline data • Significant imbalance may not affect outcome • Non-significant imbalance may affect outcome • Significance tests for baseline differences are inappropriate.

19. Significance tests for baseline differences INAPPROPRIATE Chan et al. Lancet 1997

20. Comparison of baseline data • Significant imbalance may not affect outcome • Non-significant imbalance may affect outcome • Significance tests for baseline differences are inappropriate. • Table of baseline data should focus on factors affecting outcome.

21. 45 baseline factors!

22. Comparison of baseline data • Significant imbalance may not affect outcome • Non-significant imbalance may affect outcome • Significance tests for baseline differences are inappropriate. • Table of baseline data should focus on factors affecting outcome. • Analysis adjusted for baseline factors that are known to strongly influence the outcome (Covariate-adjusted analysis). • Analysis of covariance for a quantitative outcome • Logistic regression for a binary response • Cox’s-proportional hazard model for time-to-event data

23. “Intention-To-Treat” Analysis “…results were analyzed according to the ITT principle.” Question: How were missing outcomes/ protocol violators handled in the so called “ITT” analysis?

24. “Intention-To-Treat” Analysis Endpt Savage et al. NEJM 1997

25. Recommendations for ITT Analysis • Minimize missing response on primary outcome • Follow up subjects who withdraw early • Report all deviations and missing response • Investigate & report the effect of missing response

26. Subgroup Analysis Randomised trial of home-based psychosocial nursing intervention for patients recovering from myocardial infarction. Frasure-Smith et al. Lancet 1997 “…The poor overall outcome for women, and the possible harmful impact of the intervention on women, underlie the need for…”

27. Subgroup Analysis Effect of antenatal dexamethasone administration on the prevention of respiratory distress syndrome. Am J Obstet Gynecol 1981;141:276-87.

28. Subgroup Analysis Effect of antenatal dexamethasone administration on the prevention of respiratory distress syndrome. Am J Obstet Gynecol 1981;141:276-87. Difference 6.1%

29. Subgroup Analysis Effect of antenatal dexamethasone administration on the prevention of respiratory distress syndrome. Am J Obstet Gynecol 1981;141:276-87. P value depends on effect size & SE Difference 6.1% Difference 6.2%

30. Diff in Subgroup A – Diff in Subgroup B Z = SE of the above Diff Evaluation of Subgroup Analysis Tests of interaction (assess whether a treatment effect differs between subgroups) rather than subgroup P values

31. Difference = 0.42 Difference = 0.15 Trial of vitamin D supplements in pregnancy to prevent infant hypocalcemia. BMJ 1980;281:11-4. Interaction Test No evidence that the effect of Vit D is different between bottle-fed and breast-fed infants 0.42 – 0.15 = 0.27 SE of this Diff = 0.22 Z = Diff / SE = 1.23 P = 0.2

32. General points regarding subgroup analysis • Emphasis should remain on overall comparison • More convincing if confined to a limited number of pre-specified subgroup hypothesis • Rely on interaction tests, not P values • View subgroup findings as exploratory (to be confirmed in subsequent trials)