Missing Data Due to Drop-out and Related Statistical Challenges: A Schizophrenia Study

1. Missing Data Due to Drop-out and Related Statistical Challenges: A Schizophrenia Study Suna Barlas and Mike Miller

2. Outline • Challenges of CNS trials • Study Description • Demographics • Drop out patterns and reasons • Methods for modelling the measurement process • “Sensitivity” analysis by employing different methods of estimating treatment effect; LOCF and MMRM • Methods for investigating/modelling missingness mechanism/drop-out process • Joint modelling of the two processes • Summary of findings and learnings • Future work – unending process

3. Challenges in CNS trials • Inadequate understanding of biology/disease • Lack of good or widely accepted biomarkers/surrogate markers in most cases • Measurement issues • Poorly validated/understood instruments • High placebo response • Medium- High drop out rates • Longer than “usual” trials • Some examples

4. Schizophrenia program • Wyeth conducted two Phase 2 trials testing a 5-HT2C agonist in Schizophrenia patients • The first trial was PoC, one dose performed “well” and is the subject of this presentation • The second trial was a dose-adaptive study with an active and placebo control and several doses of the test article. • High placebo response • High drop out rate • Uninterpretable results • Conclusion: discontinue the development and investigate other disease areas/indications

5. Schizophrenia PoC study description • Double-blind, randomized, parallel group study with 4 arms: low-dose, high-dose, active control, and placebo • Primary inclusion criteria: subjects with a diagnosis of schizophrenia => 1 year and hospitalized due to acute exacerbation • A 6 week study, with screening/washout + first 4 weeks in-patient, an additional 2 weeks following discharge • Primary efficacy assessment was the change from baseline in the PANSS positive subscale score at week 6 (PANSS) • The sample size was determined to be 75 subjects per study arm yielding about 80% power to detect at least a 2.5 point improvement in the mean PANSS PS, at the 10% alpha level, for either low-dose or high-dose group compared with placebo

6. Schizophrenia PoC study description • Treatment comparisons used • LOCF • ANCOVA with baseline PANSS PS as the covariate • modified ITT cohort (mITT) • mITT defined as subjects with a baseline and at least one post-baseline PANSS PS assessment • Alpha level was to be protected by testing the high dose first, then the low dose.

7. Demographics: mITT Cohort

8. Demographics: mITT Cohort

9. Primary Objective • Studies in schizophrenia tend to have high early drop out rates • This study was no exception: withdrawal before the last in-hospital week (week 4) of over 42% • Focus on the first 4 weeks of the study during which all the subjects remained hospitalized • The objective of this exercise was to learn more about factors affecting early withdrawal and efficacy • Model the last PANSS PS change from baseline and the last week when this efficacy measure was observed

10. Summary of Last Visit With a PANSS Score Through Week 4 Note: 122 out of 289 patients (42.2%) withdrew before week 4, the last scheduled in-clinic week

11. Reasons for Early Withdrawal: No Strong Imbalances Evident When Comparing Study Arm Distributions

12. Efficacy Summaries: Change from Baseline in PANSS PS Score • Each of the following summaries follows the progress of 4 cohorts/methods through the first 4 weeks (in-hospital period of the study) • 1 - all mITT patients, evaluated using MMRM • 2 – all mITT patients, evaluated using LOCF • 3 - all mITT patients with a completed week 4 evaluation, using MMRM • 4 - all mITT patients who did not have a completed week 4 evaluation, using LOCF

17. Comments on the Mean PANSS Change From Baseline • The placebo and active control responses followed a similar pattern in the sense that patients not having a week 4 assessment tended to separate early from the 4 week completers (by week 2), and these two groups tended to “bracket” the MMRM and LOCF response. • The Low-Dose responses followed a different pattern: patients not having a week 4 assessment did not separate from completers until perhaps week 4, and in general the mean responses from all analysis cohorts tended to stay together. • The High-Dose response was not as expected and never separated from placebo. • The placebo 4 week completers arguably had as good an outcome as the Low-Dose completers (-4.5 +/- 0.8 versus -5.2 +/- 0.8). • By week 4 in the MMRM analysis the mean PANSS change for placebo approached levels attained by the Low-Dose, so that the p value at week 4 was above p = 0.1. Are the usual MMRM study arm comparisons at week 4 reasonable, fair, useful in this situation?

19. More Comments on the Mean PANSS Change From Baseline • These results suggest that the risk of early withdrawal through the first 4 weeks may have been affected by treatment. • The standard descriptive summaries of sample means by visit will be affected by any selection bias and therefore could be misleading. • Since completers tend to have a greater influence on the MMRM comparison of the means at the later visits (week 4), even the MMRM comparison may be compromised by a treatment related high rate of early withdrawal (apples versus oranges at week 4)

20. Modeling PANSS Last Visit, and PANSS Change From Baseline at the Last Visit • High dropout rates possibly related to study therapy may have sabotaged the usual efficacy comparisons, but there is yet much to be learned from this study. • Further insights into this data require controlling for factors affecting both the dropout risks as well as possibly different factors affecting efficacy. • One method for jointly assessing early withdrawal and efficacy is to model both the last visit and the last PANSS change from baseline observed at this visit. • While this analysis will focus on the last observed PANSS score, this is not an LOCF analysis in the usual sense. The last observation will not be carried forward but remain with the last visit. • Risks of early withdrawal are important to identify because they provide clues regarding the overall patient efficacy and tolerance of the drug.

21. Modeling PANSS Last Visit and PANSS Change From Baseline at the Last Visit • Step 1: (Early Withdrawal) Investigate possibly treatment related factors influencing the distribution of last visits. • Step 2: (Efficacy versus Early Withdrawal) Investigate possibly treatment related factors affecting the mean PANSS change from baseline, conditioned on the patient’s last PANSS visit. • Step 3: (Overall Efficacy) From a joint model determined by 1 and 2, estimate the unconditional (marginal) mean PANSS change for each treatment by averaging the conditional mean PANSS change over the last visit distribution determined by treatment and other factors. • Step 4: (Early Withdrawal versus Efficacy) The conditional distribution of Last Visit as a function of the PANSS change is then estimated by the usual ratio of joint to marginal distribution from step 3 (Bayes).

22. Results: (Early Withdrawal) Last Visit Hazard Model, Subject Weight at Baseline Was Important

23. Results: (Early Withdrawal) Probability Distribution for Last Visit Estimated at a Baseline Weight of 65kg

24. Results: (Early Withdrawal) Probability Distribution for Last Visit Generated Estimated at a Baseline Weight = 90kg Week 4 bars 1.0 as HZD 0

25. Results: (Early Withdrawal) Probability Distribution for Last Visit Estimated at a Baseline Weight of 120kg

26. “This drug is a potent 5-HT2C full agonist” Weight gain is a problematic side effect associated with atypical antipsychotics such as (the active control), and it has been suggested that 5-HT2C antagonism is responsible for the increased weight gain. Conversely, stimulation of the 5-HT2C receptor is known to result in decreased food intake and body weight. As a result, 5-HT2C agonists would be less likely to produce the body weight increases associated with current atypical antipsychotics.

27. Some Evidence of Early Withdrawal Risk Related to Patient Weight for SCA136 – 200mg, but not for 400mg

28. Results: Last Visit Hazard Model, Subject Weight at Baseline Was Important (Early Withdrawal) • A significant decrease in early withdrawal hazard versus increasing weight was noted for the Low-Dose group (p = 0.02). • A less notable decrease in hazard versus weight was seen in the High-Dose group (p = 0.12) • Weight had little effect on the Active Control and placebo hazard for early withdrawal (p > 0.3) • The comparison of Low-Dose with placebo was significant at p = 0.02, the High-Dose group with placebo was not quite significant (p = 0.10), and the comparison of the Low-Dose group with Active Control was not significant (p = 0.2). (p values were not adjusted for multiple comparisons)

29. Results: Conditional ANCOVA Model for PANSS Change From Baseline (Efficacy versus Early Withdrawal)

30. Results: Conditional ANCOVA Model for PANSS Change From Baseline (Efficacy versus Early Withdrawal) • There was strong evidence that the last visit had an impact on the mean PANSS change from baseline • However, the mean change from baseline versus last visit was affected by study arm (treatment by last visit interaction significant at p = 0.08). • Patients completing week 4 generally had better outcomes than patients not completing week 4 (notable exception: Low-Dose group). • Difficult to compare the study arm means at each (last) study week because of possible treatment induced selection bias. • (Important Covariates) Baseline PANSS had the greatest impact on the last PANSS change from baseline versus last visit, but no other baseline covariate (particularly baseline weight) had a detectable impact on the mean change.

31. Marginal PANSS Change from Baseline Averaging Over the Last Visit Distribution (Overall Efficacy) • Using the hazard model for the last study week, a summary mean PANSS change from baseline can be computed by averaging over the last weeks 1-4. • It is possible to compute this average at selected values of important baseline assessments (in this case baseline weight had a treatment related impact on the risk of early withdrawal). • These summary means are comparable across treatments since they involve the whole cohort at fixed values of pre-treatment assessments.

32. Results: (Overall Efficacy) Averaging Over Last Visit at Selected Baseline Weights (at Baseline PANSS = 26)

33. Results: (Overall Efficacy) Averaging Over Last Visit at Selected Baseline Weights (at Baseline PANSS = 26) • The baseline PANSS score is explicit in the linear model relating change from baseline to baseline and other factors. • There is a weak trend for heavier patients at baseline to have better outcomes in the drug groups (not statistically significant), however: • The apparent effect of baseline weight on the change from baseline is implicit. Baseline weight was not part of the change from baseline linear model but was an important part of the logistic model of early withdrawal hazards. Baseline weight as an explicit parameter in the PANSS change MMRM or LOCF analysis did not yield a detectable effect. • The implicit weight effect on the PANSS change is suggestive but not conclusive or well established, since the standard errors are too large. The only statistically significant weight effect was seen for early withdrawal hazards in the Low-Dose group and its comparison to placebo.

34. Results: (Overall Efficacy) Averaging Over Last Visit at Selected Baseline Weights: Study Arm Comparisons • The estimated summary mean PANSS change for the High-Dose group had significant improvement from baseline for the selected weights > 60kg, but never separated from placebo for any weight. • The estimated summary PANSS change for the Low-Dose group had significantly better outcomes compared with placebo for all selected weights. Comparison of the Low-Dose group with Active Control yielded no statistically significant differences, but the Active Control group numerically had better outcomes at all of the selected weights. • Focusing on the average weight (90kg) and average baseline PANSS score (26), the above impressions remain, namely that the Low-Dose and Active Control groups had better outcomes compared with placebo and the High-Dose groups, with a greater separation evident compared with the usual LOCF analysis.

35. Results: Summary Mean PANSS Change from Baseline = 26, Weight = 90kg (Overall Efficacy)

36. Conditional Distribution of the Last Week given the Last PANSS Change Score (Early Withdrawal versus Efficacy) • Does the patient’s last outcome measure carry any information regarding when that outcome measure occurred? • The answer to this question provides insight into the relationship between the efficacy measure and early withdrawal, but does not answer the causality question: Was early withdrawal a result of low efficacy, or vice versa? • Methodology: compute the conditional distribution of the last visit via Bayes theorem. Here the standard deviations are used instead of the standard errors for the conditional and marginal distributions of the PANSS mean change. • The following summaries focus on the conditional probability the last visit was visit 4 (i.e. the conditional probability the patient finished the in-hospital portion of the study) given values of the last PANSS change from baseline.

37. Placebo – Minimal Baseline Weight Effect Relatively Large PANSS Change Effect (Early Withdrawal versus Efficacy)

38. Active Control – Similar to Placebo except Order Reversed(Early Withdrawal versus Efficacy)

39. Low-Dose Group – Large Weight Effect with a Smaller PANSS Change Effect (Early Withdrawal versus Efficacy)

40. High-Dose Group – Some Weight Effect Visible along with a PANSS Change Effect (Early Withdrawal versus Efficacy)

41. Probability of Finishing In-Clinic Study by Study Arm versus PANSS PS Improvement (Early Withdrawal versus Efficacy)

42. Comments Regarding Conditional Probability of Finishing In-Clinic Study given PANSS PS Change (Early Withdrawal versus Efficacy) • This type of summary provides information not immediately available from the other summaries. • Among patients showing any improvement, the PANSS change from baseline gives little additional information on the chance of finishing the study for the Low-Dose group. • The strong dependence on baseline weight was seen earlier for the Low-Dose group. • For the other study arms, particularly placebo, the probability of finishing the study increased with increasing improvement in the PANSS change from baseline

43. Conclusions • (Early Withdrawal) Baseline weight affected the risk of early withdrawal for the Low-Dose group. The lighter patients from this group had a greater risk of early withdrawal compared with placebo patients. No such effect was observed for placebo. • (Efficacy versus Early Withdrawal) The mean of the last PANSS change from baseline was strongly influenced by the week of the last visit, but this effect depended on treatment. • (Overall Efficacy) From all of the mean PANSS change scores by weight averaged over study week the Low-Dose group had significantly better outcomes compared with placebo, but the High-Dose group never separated from the placebo group. The performance of the Low-Dose group was comparable to the Active Control which numerically had the best overall efficacy response. • (Early Withdrawal versus Efficacy) The PANSS change from baseline had an effect on the early withdrawal rates for placebo, Active Control, and the High-Dose group, but not for the Low-Dose group. For all except the Low-Dose group, greater PANSS improvement increased the chance that the last in-hospital week was week 4 • Modeling the last visit and the primary efficacy assessment given the last visit has yielded some new insights and new mysteries regarding the performance of this experimental drug in the treatment of schizophrenia.

44. References • Molenberghs, G. “Incomplete data in clinical trials: analysis, sensitivity, and sensitivity analysis”, Drug Information Journal; July, 2009, vol 43, pp. 409-429. • Muthen, B, Masyn, K, “discrete time survival mixture analysis” J. Educ. Behav. Stat.; 30, no. 1, Spring 2005, pp 27-58. • Xekalaki, E., “Hazard functions and life distributions in discrete time”, Commum. Statist. – Theor. Meth. 12(21), 1983, pp 2503-2509.

45. Back up slides

46. Modeling PANSS Last Visit, and PANSS Change From Baseline at the Last Visit • Let W = last week the PANSS score was observed. • Let DLT = observed PANSS change from baseline at this last visit. The joint probability density function (pdf) of W, DLT has the form: J(w,dlt) = f(dlt| w, C1, θ1) m(w| C2, θ2) where the last week w is a covariate, C1 is a set of other covariates (including study arm, baseline PANSS score), and θ1 is a set of population parameters to be estimated from the observed (dlt, w, C1). Similarly, C2 is a possibly different set of covariates related to w (possibly including study arm), and θ2 is a different set of population parameters to be estimated from the observed (w, C2) (Simplified form of the general pattern mixture model – Molenberghs (1))

47. Modeling PANSS Last Visit • The following general rate model tends to fit the observed last visit distributions from this study: • Let W be the last week where a PANSS score was obtained. Then: Prob[W ≥ w] = λ(w-1) for weeks 1 through 4, 0 < λ < 1 Here the modified ITT cohort having at least one post-baseline score is used, so Prob[W ≥ 1] = 1.0 . (All mITT subjects had a week 1 evaluation) • Hence, taking successive differences: m(w| λ) = λ(w-1) (1 – λ) for w = 1, 2, 3 = λ(4-1) = λ3 for week 4. Note: If h = -ln( λ), then Prob[W ≥ w] = e- h(w-1) analogous to exponential survival with constant hazard h. (Alternative terminology has the constant hazard as 1 – λ. See Muthen and Masyn (2) , Xekalaki (3) )

48. Modeling PANSS Last Visit • The advantage of this model is the use of a single parameter to explain the observed distribution of last visits over the study week. • A logit transformation was used to estimate the rate constant as a function of observed covariates: logit(λ) = C2’ θ2 • Maximum likelihood estimates were obtained for θ2 , and those factors having a notable impact on λ were retained in the final model. Results were expressed in terms of the hazard h = - ln(λ). • Estimates of h were obtained for each study arm at selected values of important covariates. These estimates were then converted to probability distributions over the study weeks 1, 2, 3, 4. • Baseline factors screened included, study arm, age, sex, race, baseline PANSS score, weight.

49. Results: Last Visit Logistic Model (Early Withdrawal) • Several baseline covariates were screened, and baseline weight was found to have a notable effect on the early withdrawal rates. Here are more details regarding the form of the weight model for early withdrawal. • Recall that the withdrawal hazard h was defined by: Prob[last week => j] = exp(-h (j-1) ), where h is assumed constant for weeks 1,2,3,4, and all subjects had a PANSS score at least at week 1. • A logit linear model relating the effects of study arm and baseline weight to the hazard is: logit( exp(-h) ) = mu + a (study arm) + b (baseline weight) + c (study arm x weight) Maximum likelihood estimates were obtained for the study arm, weight, study arm by weight model parameters which were converted to MLE’s for slopes and intercepts for each study arm. Estimated probability distributions determined by weight and study arm were then generated. (slides 23-28)

50. Modeling Change from Baseline in PANSS at the Last Visit • The usual analysis of covariance model was used: DLT(PANSS) = u + x’ + trt’a + lstwk’b +( trt*lstwk)’c +  Z, where x is a vector of baseline covariates (including baseline PANSS), trt repesents the 4 study arms, lstwk = last visit (weeks 1, 2, 3, 4), and trt*lstwk are the interaction terms. • Estimates of the least squares mean DLT were obtained at each study week for each study arm, possibly at selected values of important covariates. • Baseline factors screened included, study arm, age, sex, race, baseline PANSS score, weight. (slides 29, 30)