Multi-Clinic and Multi-National Trials: A regulatory perspective

1. Multi-Clinic and Multi-National Trials: A regulatory perspective Charles Anello Sc.D. Deputy Director, Office Of Biostatistics Office of Pharmacovigelence and Statistical Sciences Center for Drug Evaluation and Research anello@cder.fda.gov ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

2. Acknowledgements I would like to express my appreciation to Robert T. O’Neill, Satya Dubey, Jim Hung, Robert Temple and John Lawrence for their helpful suggestions in the preparation of this talk. ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

3. Disclaimer The views expressed are those of the author and do not necessarily represent those of the Food and Drug Administration. ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

4. Topics • Motivation • Regulatory Context • Changing Context • Some statistical Issues • Multi-national studies • An example • Conclusions ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

5. Motivation • University Group Diabetes Program • Coronary Drug Project • How did multi-clinic and multi-national trials become so common? ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

6. Regulatory Context Kefauver-Harris Amendments to FD &C Act. • The need to establish efficacy based on substantial evidence • Adequately powered studies are needed to detect clinically meaningful drug effects • As the focus shifted to modest effects the need for large enough studies to establish efficacy led to the increase use of multi-clinic or multi-investigator trials. ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

7. Changing Context • Format and Content Guidance • ICH E9 • Evidence Document • Emergence of a global pharmaceutical industry and the multi-nationalTrial ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

8. Format and Content Guidance • Results for individual studies should be presented • Suggests test for interaction between treatments and centers • No guidance on how this test is to be conducted or the significance level to be used ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

9. Format and Content Guidance • Extreme or opposite results need to be discussed • Demographic , baseline and results presented by center • Graphic displays by center on key characteristics and findings • Special attentions is given to age, gender, ethnicity (or region). ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

10. Why the emphasis on interaction? • The review of a clinical trial as several aspects: • Are the treatment groups comparable • Are the potential sources of bias that may account for the findings • Is the treatment effect consistent across the centers ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

11. Why the emphasis on interaction? • The review function as a detective aspect • The review function as a quality control aspect • The goal of the review is to establish efficacy and safety and provide information to physicians in the product label about how and for whom the drug works • If subgroups (e.g., clinics) show different results( i.e., interaction) this could impact the label ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

12. ICH E9 • Suggests multi-clinic trials are needed to speed up the time needed to conduct trials • Provide a basis for generalization • Stress the need to implement the common protocol in a similar manner in all clinics across all regions • Offers advice how to approach the analysis ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

13. ICH E9 • Test main effect first , if significant • Test interaction as an exploratory analysis • If there are a large number of centers, it is less important to consider interaction. ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

14. Evidence Document • When only one multi-clinic trial is available for establishing substantial evidence of efficacy and safety • The results need to be statistically very persuasive • Internally consistent ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

15. Evidence Document • No single site, clinic provides an unusually large fraction of patients • No single site, clinic is responsible for a disproportionate favorable effect • Estimated effects are consistent across clinics • Multiple endpoints are consistent • Results highly statistically significant p<<.05 (eg. .001) ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

16. Lack of consistency in statistical advice? • In an ANOVA situation the Format and Content document suggest one tests first for interaction but doesn’t say how to do it. • ICH E9 suggests one test main effects first only they are significant does one examine interaction • Evidence document focuses on the need for internal consistency but leaves open how consistency is to be established. No advice on statistical approach ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

17. Looking at Interaction • There is no consistent approach • Some advocate fixed effect models • Some advocate random effect models • Some advocated mixed effect models • And so on. ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

18. Some statistical Issues • Are centers fixed or random • If an ANOVA is used would one use the type II or type III sums of squares • Other approaches ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

19. Are Centers Fixed or Random? ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

20. Centers fixed or Random • The decision to call centers fixed or random has implications for the analysis Arguments in support of fixed effects: • only approach for single center • gives precise answer to well defined question • only realistic approach with few centers, • definition of center is arbitrary Senn (1998) ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

21. Centers fixed or Random • In support of the Random effect model • approximate answer to difficult question • affect more general • wider confidence interval Senn (1998) ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

22. Are Centers Fixed or Random? There are other opinions Fleiss(1988) favored fixed effect model with a test for interaction Grizzle(1988) questioned the fixed effect model and preferred the random effect model Senn(1993) remained open minded on this issue ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

23. Are Centers Fixed or Random? • The random effect model incorporates interaction into the error term. • The fixed effect model allows one to explore interaction if it exists • Both approaches can be justified at some level. ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

24. Are Centers Fixed or Randon • In order to label a product correctly one needs to be sure the effect is consistent across all relevant subgroups • From my perspective the fixed effect model seems more relevant. • Whereas the random effect model may answer a question it may answer a different question? • It may give the right answer to the wrong question. Kimbell(1965) would have called this a “type III error” ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

25. What is the correct Sums of Squares in an ANOVA analysis • The choice is between the type II and typeIII SS for SAS GLM • Many authors support the type II SS which involves weighting by center size • FDA has been alleged to favor the type III SS • While I can not speak for all FDA I think the current view within CDER is with those supporting the type II SS ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

26. Much Discussion about variation between centers • Is to be expected but difficult to detect • Is it quantitative (i.e., same direction only magnitude differs) very common • Is it qualitative (i.e., treatment shows benefit in some centers , Placebo shows benefit in others) less common • Qualitative interaction is of concern but not found very often and hard to establish Peto(1982) ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

27. Expected Center Variation • For fixed sample size as the number of centers increases so does the chances of reversal • It has been shown that with as few as six centers there is better than a 50% chance of having at least one center where the placebo effect would exceed the treatment effect. O’Neill et. al.(1998) ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

28. Multi-national Differences Temple (SCT 2003) discussed the interpretation of international differences in clinical trail results I will focus on a single international trial in which a subgroup( country or region) appeared to show a result different from the overall finding of the study ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

29. Multi-national Multi clinic Trials • MERIT-HF Trial a multi-national trial of mortality in patients with chronic heart failure. • When the regions of US versus Europe was examined by a post hoc analysis differences were observed although not quite at the “qualitative difference” • one group showed a benefit another did not. ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

30. MERIT-HF Trial • Metoprolol succinate extended release (Toprol XL) the MERIT-HF Trial • Placebo controlled, multi-national 3991 patient trial in NYHA Class II_III CHF ,treated added to ACEI, digitalis and other Rx • Endpoints: all cause mortality and all cause mortality plus hospitalization ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

31. MERIT-HF Trial • The differences occurred in the mortality findings. The combined endpoint showed a consistent finding whereas the mortality endpoint showed a 50% reduction in the Non US population compared to a 5% excess in the US population. With a post hoc p-value of .003. ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

32. MERIT-HF Trial • The regions were not a pre-specified subset • Approval was not an issue the study showed a benefit for mortality plus hospitalization • The agency’s internal debate focused on “full disclosure” this apparent unexplained qualitative interaction in a component of composite endpoint. • Whether regions matter in this instance is an open question. Temple(2003) ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

33. Summary • Multi-clinic trials are an important aspect of drug development and regulatory review • Multi-clinic multi-national trials are becoming more important • From a regulatory perspective consistency in results are important • If interaction occur, they need to be identified and understood • Interactions may have implications for the product label and public health ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

34. Summary • Some statistical tools for dealing with these interaction exist, but may have limited utility due to low power • The fixed effect model may be more useful when there are not to many clinics and the sample sizes are large and approximately equal size and it is reasonable to assume a constant effect across the clinics • The random effect model may be more useful when there are large number of clinics with few observations per clinic and one can not address the question of center by treatment interaction. • When an ANOVA is appropriate, the type II sums of squares is often the Sums of squares of choice ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD

35. Summary • Quantitative and Qualitative interactions happen although Qualitative interaction is rare • Observed clinic or regional differences need to be noted, explained if possible and reported but not necessarily believed. • The potential for regional differences, may have implications for study design and the way DSMBs monitor trials. ASA-Industry Workshop,Sept.17-19,2000 Bethesda, MD