Emerging Issues with Adaptation of Clinical Trial Design in Drug Development *

1. Emerging Issues with Adaptation of Clinical Trial Design in Drug Development* H.M. James Hung Division of Biometrics I, Office of Biostatistics, OPaSS, CDER, FDA Presented in FDA/Industry Workshop, Bethesda, Maryland, September 16, 2005 *The view expressed here is not necessarily of the U.S. Food and Drug Administration.

2. Contributors Sue-Jane Wang (OB/OPaSS/CDER/FDA) Robert O’Neill (OB/OPaSS/CDER/FDA) John Lawrence (DB1/OB/OPaSS/CDER/FDA) Acknowledgment Thanks are due to Charles Anello for valuable comments. This presentation is partly based on the research, supported by FDA/CDER RSR funds, #05-02, #05-14, 04-06, 02-06, 99/00-008, 96-010A. J.Hung, 2005 FDA/Industry Workshop

3. Outline • Design consideration • Logistics issues • Statistical design/analysis issues • Regulatory experiences • Utility and pitfalls of design adaptation • Summary J.Hung, 2005 FDA/Industry Workshop

4. Design Consideration • As a guiding principle, for Phase III trial, fixed information design without any possibility of early stopping is preferred. • minimize operational bias • valid statistical test available • unbiased estimator of treatment effect available • valid confidence interval available These are the keys to the quality of statistical inference vital to ability of drawing conclusion from trial data. J.Hung, 2005 FDA/Industry Workshop

5. Design Consideration Flexible designs Group-sequential design adaptive design Phase 2/3 combination design ………….. Utility of flexible designs in Phase II trials is probably obvious, at least for exploratory purpose, without subjecting to too many false positives. J.Hung, 2005 FDA/Industry Workshop

6. Design Consideration Example: Surrogate markers are used in 1st phase of a clinical trial for predicting clinical effect or changing design in 2nd phase, e.g., - changing clinical endpoints - dropping or adding treatment arms - re-estimating sample size - enriching the patient population Clinical benefits are assessed at the end (second phase) of the trial where design specifications may have been modified. J.Hung, 2005 FDA/Industry Workshop

7. Design Consideration • When flexible design is considered, we need to ask: • How to proceed logistically? • - standard operation procedure (SOP) • - ability to comply with SOP • - responsibility of every participant • Are trial conducts reviewable by regulatory • agency? • Goal: minimize operational bias J.Hung, 2005 FDA/Industry Workshop

8. Design Consideration • When flexible design is considered, we need to ask: • Is valid statistical test available? • Is unbiased estimator of treatment effect • available? • Is valid confidence interval available? J.Hung, 2005 FDA/Industry Workshop

9. Logistics Issues • Standard Operation Procedure • charters of all parties including independent • thirty party (e.g., DSMB, CRO) • who see what? • how to ensure compliance with SOP? • how to minimize possible influence of • adaptation on investigator/patient behavior? • will knowledge of external trials have adverse • influence on the current trial? J.Hung, 2005 FDA/Industry Workshop

10. Statistical Design/Analysis Issues • Much research has provided valid statistical • methods for analysis of primary endpoint in • flexible design trials • - what point estimate and interval estimate to • report in drug label needs more attention • (have concern about using these estimates for • future superiority/NI trials) J.Hung, 2005 FDA/Industry Workshop

11. Statistical Design/Analysis Issues • Unclear how to best analyze important • secondary endpoints after primary endpoint is • positive • Unclear whether validity of multiplicity • adjustment still holds • Unclear how to incorporate design efficiency • consideration into entire drug development • program • - involve prediction of benefit/risk ratio • - per-trial statistical efficiency is insufficient J.Hung, 2005 FDA/Industry Workshop

12. Regulatory Experience #1 In an oncology trial, time to progression (TTP) was a primary surrogate endpoint and overall survival (OS) was the ultimate primary endpoint. Group-sequential design was employed with an alpha-spending function for TTP. After TTP was highly significant at an interim time and the trial was stopped, how to test OS given that only 20% of the planned total deaths is available? - May use the spending function of TTP for OS J.Hung, 2005 FDA/Industry Workshop

13. Regulatory Experience #2 • In a heart failure trial, the primary endpoint is a composite endpoint of clinical outcomes (death, MI, stroke, etc.). Sample size and total # of events are increased, based on a new projected effect size from the data of an interim analysis. • How to test the component endpoints and other secondary endpoints? • relationship in location parameters of and • correlation between the endpoints need to be • considered J.Hung, 2005 FDA/Industry Workshop

14. Regulatory Experience #3 • A trial is designed to study mortality endpoint. • It is then divided into two subtrials to study a non-mortal endpoint (e.g., quality of life). • -alpha allocation • - replication concept • - complication with interim analysis • - p-value adjustment for the non-mortal endpoint • - how to handle competing-risk problem J.Hung, 2005 FDA/Industry Workshop

15. Regulatory Experience #4 Surrogate markers or biomarkers are used in early stages for predicting drug effect or changing design elements in later stages, e.g., - changing clinical endpoints - dropping or adding treatment arms - re-estimating sample size - enriching the patient population Clinical benefits are assessed at the end of the trial whose design specifications may have been modified. J.Hung, 2005 FDA/Industry Workshop

16. Regulatory Experience #4 <cont’d> • Q: Can clinical endpoint data from exploratory stages be combined with Phase III data in analysis for confirmatory purpose? • If yes, how to proceed in analysis? • - Weighted combination Z statistic and the • corresponding point estimate and CI can be • useful tools. • Need to prespecify the weight ! • Concept of overall type I error can be unclear • Combining two stages at best results in one • study J.Hung, 2005 FDA/Industry Workshop

17. Regulatory Experience #4 <cont’d> - Issue with # of hypotheses to explore in Phase II (pertaining to statistical control of total type I errors)? No (or ignored) if Phases II and III are not combined Yes (needing attention) in Phases II and III are combined This makes little sense. [Hung, Wang, O’Neill (2004)] How to proceed logistically? J.Hung, 2005 FDA/Industry Workshop

18. Utility and Pitfalls of Design Adaptation Major pitfalls concern compromise of integrity of trial conduct. Major utility is potential saving of a trial if the planning assumptions fall short. Flexibility can enhance the probability of trial success with integrity protected if properly planned and complied. J.Hung, 2005 FDA/Industry Workshop

19. Utility and Pitfalls of Design Adaptation For statistical analysis, with proper determination of weighting, a weighted combination of Z statistics can be attractive and powerful. For any fixed weight, Z is valid under the null hypothesis. A consistent estimator and valid confidence interval for the parameter Z tests are available to draw conclusion consistentwith that from Z. How to find a good w? J.Hung, 2005 FDA/Industry Workshop

20. Utility and Pitfalls of Design Adaptation If the weight w is random, then with properly adjusted critical value, Z can be valid. Q: Are point estimator and CI available to enable us to draw conclusion consistent with that from Z? J.Hung, 2005 FDA/Industry Workshop

21. Utility and Pitfalls of Design Adaptation Don’t rely too much on the conditional power evaluated at the interim observed estimate or any data observed at too early interim looks - planning a trial conservatively is still needed. J.Hung, 2005 FDA/Industry Workshop

22. Summary • More careful planning is necessary for flexible design (group-sequential or adaptive) in regulatory applications - operational bias is a critical issue • Conventional statistical methods need tune up in use of flexible designs - adjust critical value of statistical test - find valid point estimate and CI J.Hung, 2005 FDA/Industry Workshop

23. Summary • Analysis methods suiting flexible designs may be preferable to properly adjusted conventional methods - need to explain the differences to our clients (can be very difficult) - still need valid point estimate and CI J.Hung, 2005 FDA/Industry Workshop

24. Summary • Too much adaptation should be discouraged - what hypothesis to test becomes unclear - protecting integrity of trial is difficult - what infrastructures are needed for logistics is still quite unknown - reviewability by regulatory agency is unknown and very difficult J.Hung, 2005 FDA/Industry Workshop

25. Selected References Bauer, Köhne (1994, Biometrics) Bauer, Röhmel (1995, Stat. In Med.) Lan, Trost (1997, ASA Proceedings) Proschan, Hunsberger (1995, Biometrics) Fisher (1998, Stat. In Med.) Shen, Fisher (1999, Biometrics) Cui, Hung, Wang (1999, Biometrics) Bauer, Kieser (1999, Statistics in Medicine) Posch, Bauer (1999, Biometrical J.) Kieser, Bauer, Lehmacher (1999, Biometrical J.) Lehmacher, Wassmer (1999, Biometrics) Müller & Schäfer (2001, Biometrics) Hellmich (2001, Biometrics) Liu & Chi (2001, Biometrics) J.Hung, 2005 FDA/Industry Workshop

26. Bauer, Brannath, Posch (2002, Method Inform Med) Brannath, Posch, Bauer (2002, JASA) Cui (2002, Encyclopedia of Biopharmaceutical Statistics) Li, Shih, Xie, Lu (2002, Biostatistics) Lawrence (2002, J. of Biopharmaceutical Statistics) Lawrence (2002, J. of Pharmaceutical Statistics) Lawrence & Hung (2002, Biometrical J.) Jennison & Turnbull (2003, Statistics in Medicine) Tsiatis & Mehta (2003, Biometrika) Liu, Proschan, Pledger (2003, JASA) Proschan, Liu, Hunsberger (2003, Statistics in Medicine) Wang, Hung, O’Neill (2004, ASA Proceedings) Hung, Wang, O’Neill (2004, ASA talk) Liu (2002, 2003, 2004, ASA talks, other talks) Lan (2000-2005 ASA, ICSA, FDA talks) Mehta (2004, talk on FDA-MIT workshop) J.Hung, 2005 FDA/Industry Workshop

27. Hung, Wang, O’Neill (2004, BASS talk) Lan, Soo, Siu, Wang (2005, J. of Biopharmaceutical Statistics) Hung, Cui, Wang, Lawrence (2005, J. of Biopharmaceutical Statistics) Hung (2005, WNAR talk) Wang, Hung (2005, ASA talk) Liu (2005, WNAR talk) J.Hung, 2005 FDA/Industry Workshop