Robert T. O’Neill , Ph.D. Director , Office of Biostatistics CDER, FDA

1. Considerations Regarding Choice of the Primary Analysis in Longitudinal Trials With Dropouts: An FDA Perspective Robert T. O’Neill , Ph.D. Director , Office of Biostatistics CDER, FDA Presented at the FDA /Industry Workshop; Bethesda, Maryland; September 17-19, 2003

2. Disclaimer The opinions expressed are mine and do not represent CDER policy on this issue. Ongoing research within CDER is directed toward more specific guidance which will supplement that contained in the ICH E9 Guidance “Statistical Principles for Clinical Trials

3. Outline • Issues about the dropout process in longitudinal clinical trials : focus on informative, treatment related missingness • Terminology used to express missing data • The literature: Problem formulation and approaches - Different research approaches to the issues • Some general conclusions from this research • Concluding remarks on choosing a primary analysis and its pre-specification when the analysis may depend upon the data pattern

4. Issues • Regulatory setting places emphasis on confirmatory studies, pre-specification of objectives, hypotheses, analyses, documentation • How to specify in the protocol the primary strategy for dealing with missing data - if you presume that it will be informative - and you have not observed the data yet • Considerable literature on the matter, unclear as to which approach to choose, when and why, and the considerable computational efforts involved • Documentation and reporting of the criteria, choice of approach (model), and certainty of the conclusions

5. Patient withdrawals from treatment exposureWhy do subjects stay in clinical trials, why do they withdraw from assigned therapy, when do they withdraw, and how do they differ from completers ? • Efficacy (lack of or benefit from) • Safety (toxicity, tolerability, neither) • Both • Aggravation of the trial • When in time do they leave therapy in trial (short duration, long duration) • Symptomatic relief vs unperceived benefit

6. What is the shape of response time patternEfficacy and Toxicity • If toxicity is time dependent, and efficacy is time dependent, withdrawal due to either side effects or perceived early efficacy, censors the efficacy outcome whose time dependency may follow different pattern • Bivariate outcome (Efficacy (t), Safety(t))

7. Four situations (E,S) at time T E S(T) Patient perceives effective response AND no toxicity occurs E S(T) Patient perceives effective response AND experiences toxicity • may or may not leave depending upon tolerance and preferences E S(T) Patient perceives No effective response AND no toxicity occurs E S(T) Patient perceives No effective response AND experiences toxicity • may or may not leave but is MORE likely to leave than patient above who perceives effective response

8. The problem formulation in the literature ?Which to choose as primary ? • Likelihood based methods • mixed model repeated measures (MMRM) • Pattern -mixture models • Selection models • Ad hoc methods (LOCF, worst case imputation) • Single and multiple imputation approaches • The terminology

9. Terminology for missing data • MCAR - missing completely at random • MAR - missing at random • MNAR - Missing not at random • NIM - Non - Ignorable Missingness • OC observed cases • AAD - all available data • CC - complete cases • CAR - coarsened at random • LVCF (last value carrying forward) - LOCF • Dependent dropouts - DD

10. More Terminology for missing data • Classification of drop-out mechanism • independent of data - IDA • dependent on observed data - ODA • dependent on missing data - MIDA • dependent on observed and missing data - OMI • dependent on observed data and covariates - ODACO

11. MADMissing and Deferred (or Delayed):Another new term • MAD - missing and deferred • at random • completely at random • completely uninterpretable

14. What is unique about missing data in clinical trials ? • Monotonically Missing data is potentially an outcome by itself • Why ? - It can be a surrogate for patient preference, acceptability with therapy, and can potentially be unpredictive of where the subject would be in the future (where no observations are taken or available) • With monotone missing data, the ‘dropout mechanism’ is very likely informative • Possible to plan to collect information during study prior to treatment withdrawal and prior to study completion but post treatment withdrawal (conditioning)

15. Example of longitudinal Efficacy response/score by visit - missing for toxicity/safety reasons not considered Test Baseline Control Higher is bad 1 2 3 4 5 Visit

16. Are slope and baseline predictive of how long a patient stays in trial ? No treatment effect Test Baseline Control Higher is bad 1 2 3 4 5 Visit

17. Is baseline predictive of how long a subject stays in trial Treatment relationship is less clear Test Control Baseline Higher is bad 1 2 3 4 5 Visit

18. Maximum likelihood approachesFactoring the likelihoodInformation is in the conditioning

19. The observed data and the missing dataWhich do you wish to condition on and why ? Shared parameter models not considered here, eg. same parameter in observed and missing data models

20. Selection Model • Factors the joint distribution of the observed data Y and the missing data M into the marginal distribution of the observed response times M the conditional distribution of missingness given Y = y Heitjan, Ignorability and Bias in clinical trials; Stat Med 18, 2421-2434 (1999)

21. Selection Model When the missing data are NOT IGNORABLE, one has to specify an explicit model for the missing-data mechanism M to make an appropriate inference for theta ,

22. Selection Model-What to do if non-ignorable occurs - • Specify an explicit model for the missing data mechanism • Wu and Carroll for slope analysis in longitudinal studies • Do sensitivity and robustness analysis, under different plausible models for the missing data mechanism

23. Pattern-mixture models • Factors the joint distribution of Y and M as the marginal distribution of M (the missing pattern) times the conditional distribution of observed data Y given M = m (the response given missing pattern ) • The data are stratified by missing data patterns and inference is on the conditional model parameter 

24. Pattern - mixture models The first factor says that the data are stratified by missing data patterns and the inference is given to the conditional model parameter 

25. Choice of missingness models • Pattern-mixture model: patterns of response, distribution of effects within patterns - need for a lot of data in each pattern - information in number of subjects in each pattern • Selection model : response and dropout distribution - very dependent upon assumptions of models • Shared parameter models • How much missingness (% of total N) can be tolerated ? • Overall • Between treatment groups • Early, middle, later in the study • Reasonable imputation strategies, including LOCF

26. Step 1: Model Selection ? • MMRM • Selection modeling - the second factor corresponds to the self selection of individuals into “observed” and “missing” groups. • Pattern- mixture models - a mixture of different populations, characterized by the observed pattern of missingness • e.g. 4 times measured; subjects with 1,2,3,4 measurements form the four patterns • estimate treatment effects within patterns and then combine in some way

27. Step 2 : Sensitivity Analysis - Which strategy, when , why • Determining evidence for MNAR • Model correctness (selection, pattern-mixture) • Local Influence of patients on power and detection ability • Documenting the strategies and what was done • Supporting the range of possible conclusions consistent with the data

28. Some selected literaturecomparing different strategies

29. Three papers on comparing performance of competing analysis strategies 1. 2. 3. Analysis of Longitudinal Clinical Trial Incomplete Data ; O. Siddiqui and J. Hung

30. Compares impact of fixed value imputation (FVI) like LOCF with m.l. general linear models of the observed data

31. Uses a Pattern - mixture model to make inference on the unconditional, hypothetical complete-data mean Non zero values to have dropouts have outcomes that are worse

32. Miller, Morgan, Espeland, Emerson Message • Variability of the measurements needs to be addressed • The direction of bias, by not accounting for it, is not predictable • Under several non-ignorable non-response scenarios, m.l. based analyses can yield equivalent hypothesis tests to those obtained when analyzing only the observed data.

34. Consideration jointly of: • 2 arm trial, with change in spine deformity index (SDI) over a 4 year duration with measurements at each of 4 years • Linear progression of disease • Disease progression mechanisms (early, middle , late) • Dropout mechanisms • Considers each subject’s last observation is dependent on prior repeated measurement • 14 separate methodological strategies for dealing at the analysis with missing data

37. Large black dot is unacceptable power reduction

38. Large black square is inflation of type 1 above 7.5%

39. Message The adequacy of a strategy for dealing with missing values strongly depends on whether the courses of disease are similar or very different in the study groups. Therefore knowledge about the courses of longitudinal data is important besides information on drop- out rates for planning an adequate ITT analysis. If the information about the courses of disease is not available at the planning stage of a clinical trial, the ICH E9 guideline suggests correcting the strategy for dealing with missing values in a blind review stage before analysis of the trial starts Pre- definition of methods [ of dealing with missing values ] may be facilitated by updating this aspect of the analysis plan during the blind review . Thus, the blind review is a possibility to get an idea about the patterns of courses of the endpoint, thus making the choice of an adequate strategy easier. However, a blind review including the main endpoint might induce problems if obvious treatment effects show up at this stage, giving away treatment groups. For judging the adequacy of an approach for dealing with missing values, information about rates and times of drop- outs as well as courses of disease must be provided in the publication of the results.

40. Messages • For drop-out rates less than 20% AND similar courses of disease in the treatment groups, missing values might be replaced by mean of other groups • For larger drop-out rates OR less similar course of disease, no adequate recommendations can be given • Type 1 error increases drastically for the different strategies, especially if the course of disease vary between treatment groups • There is is NO strategy which is adequate for all different combinations of dropout mechanisms, drop-out rates or less similar courses of disease and no adequate recommendations can be given.

41. Presuming informative missingness - what to do ? Computational burden is the issue

42. Adjusting for Non-ignorable Drop-out Using Semiparametric Nonresponse Models Sharfstein, Rotnitzky and Robins, JASA,V 94; 1096-1120 (1999) See Commentaries pages 1121-1146

43. Comparison of estimation methods

45. Wei, L and Shih, WJPartial imputation approach to analysis of repeated measurements with dependent dropouts Statist. Med. 2001; 20: 1197-1214

46. 3 Conditions 1. The drop-out rates are the same in both treatment groups 2. Dropout process depends on the outcome variable in the same manner in both treatment groups 3. Common variances for the outcome variable in both treatment groups

47. The Wei and Shih approach is to control C1, so that the dropout rates become the same (or nearly similar) after partially imputing those needed to made the rates the same.

48. One can condition only on what was observed and measured - Its effectiveness depends on what you know in advance to condition on • Dropout rates in each treatment group • Same or different • Time pattern same or different • How many identifiable cause specific reasons for dropouts , and are they the same or different in each treatment group • Example: ES, E S, E S, E S

49. Two papers on joint analysis of dropout as a response and observed repeated data