Exploratory Analyses Aimed at Generating Proposals for Individualizing and Adapting Treatment

1. Exploratory Analyses Aimed at Generating Proposals for Individualizing and Adapting Treatment S.A. Murphy BPRU, Hopkins September 22, 2009

2. Outline • Why Adaptive Treatment Strategies? • “new” treatment design • Constructing Strategies • Why SMART experimental designs? • “new” clinical trial design • Q-Learning & Voting • Example using CATIE

3. Adaptive Treatment Strategies operationalize multi-stage decision making. • These are individually tailored sequences of interventions, with intervention type and dosage adapted to the individual. • Generalization from a one-time decision to a sequence of decisions concerning interventions • Operationalize clinical practice. • Each decision corresponds to a stage of intervention

4. Why use an Adaptive Treatment Strategy? High heterogeneity in response to any one intervention What works for one person may not work for another What works now for a person may not work later Improvement often marred by relapse Remitted or few current symptoms is not the same as cured. Co-occurring disorders/adherence problems are common 4

5. Example of an Adaptive Treatment Strategy Drug Court Program for drug abusing offenders. Goal is to minimize recidivism and drug use. High risk offenders are provided biweekly court hearings; low risk offenders are provided “as-needed court hearings.” In either case the offender is provided standard drug counseling. If the offender becomes non-responsive then intensive case management along with assessment and referral for adjunctive services is provided. If the offender becomes noncompliant during the program, the offender is subject to a court determined disposition.

6. The Big Questions • What is the best sequencing of treatments? • What is the best timings of alterations in treatments? • What information do we use to make these decisions? • (how do we individualize the sequence of treatments?)

7. Outline Why Adaptive Treatment Strategies? “new” treatment design Constructing Strategies Why SMART experimental designs? “new” clinical trial design Q-Learning & Voting Example using CATIE 7

8. Short Term Decision Making • In short term decision making, decision makers use strategies that seek to maximize immediate rewards at each stage of treatment. Problems: • Ignore longer term consequences of present actions. • Ignore the range of feasible future actions/interventions • Ignore the fact that immediate responses to present actions may yield information that pinpoints best future actions

9. Basic Idea for Constructing an Adaptive Treatment Strategy: Move Backwards Through Stages. (Pretend you are “All-Knowing”)

10. Why SMART Trials? What is a sequential multiple assignment randomized trial (SMART)? These are multi-stage trials; each stage corresponds to a critical decision and a randomization takes place at each critical decision. Goal is to inform the construction of adaptive treatment strategies.

13. Alternate Approach • Why not use data from multiple trials to construct the adaptive treatment strategy? • Choose the best initial treatment on the basis of a randomized trial of initial treatments and choose the best secondary treatment on the basis of a randomized trial of secondary treatments.

14. Delayed Therapeutic Effects Why not use data from multiple trials to construct the adaptive treatment strategy? Positive synergies: Treatment A may not appear best initially but may have enhanced long term effectiveness when followed by a particular maintenance treatment. Treatment A may lay the foundation for an enhanced effect of particular subsequent treatments.

15. Delayed Therapeutic Effects Why not use data from multiple trials to construct the adaptive treatment strategy? Negative synergies: Treatment A may produce a higher proportion of responders but also result in side effects that reduce the variety of subsequent treatments for those that do not respond. Or the burden imposed by treatment A may be sufficiently high so that nonresponders are less likely to adhere to subsequent treatments.

16. Diagnostic Effects Why not use data from multiple trials to construct the adaptive treatment strategy? Treatment A may not produce as high a proportion of responders as treatment B but treatment A may elicit symptoms that allow you to better match the subsequent treatment to the patient and thus achieve improved response to the sequence of treatments as compared to initial treatment B.

17. Cohort Effects • Why not use data from multiple trials to construct the adaptive treatment strategy? • Subjects who will enroll in, who remain in or who are adherent in the trial of the initial treatments may be quite different from the subjects in SMART.

19. Examples of “SMART” designs: • CATIE (2001) Treatment of Psychosis in Alzheimer’s Patients • CATIE (2001) Treatment of Psychosis in Schizophrenia • STAR*D (2003) Treatment of Depression • Pelham (on-going) Treatment of ADHD • Oslin (2009) Treatment of Alcohol Dependence

20. Constructing proposals for more deeply tailored adaptive treatment strategies: Q-Learning Q stands for “Quality of Treatment” Q-Learning is a generalization of regression to multistage treatment

22. In Q-Learning we run a regression at each stage, moving backwards through the stages.

23. Clinical Antipsychotic Trials of Intervention Effectiveness (Schizophrenia) • Multi-stage trial of 18 months duration • Relaxed entry criteria • A large number of sites representing a broad array of clinical settings (state mental health, academic, Veterans’ Affairs, HMOs, managed care) • Approximately 1500 patients

24. CATIE Randomizations(simplified) Stage 1 Randomized Treatments OLAN QUET RISP ZIPR PERP Stage 2 Treatment preference Efficacy Tolerability Randomized Treatments CLOZ OLAN QUET RISP OLAN QUET RISP ZIPR Stage 3 Treatments selected many options by preference

25. Constructing Dynamic Treatment Regimes using CATIE • Reward: Time to Treatment Dropout • Stage 1 regression analysis: • Controls: TD, recent exacerbation, site • Tailoring variable: pretreatment PANSS • Stage 2 regression analysis: • Controls: TD, recent exacerbation, site • Tailoring variables: “treatment preference,” stage 1 treatment, end of stage 1 PANSS Exploratory Analyses

26. Voting • (exploratory analysis) • Our goal is to estimate the probability that a treatment would look best if we repeat the CATIE study. We want to estimate this chance for each treatment at each phase. • We “simulate” the action of repeating the study using bootstrap samples. Each bootstrap sample “votes” for the treatment it finds best at stages 1 and 2. The fraction of votes for a treatment is the estimate of the probability that this treatment will be found best. 26

29. Challenges • We have since improved the voting and can now add confidence intervals. • Clinical Decision Support Systems • We need to be able construct adaptive treatment strategies that recommend a group of treatments when there is no evidence that a particular treatment is best.

30. Acknowledgements: This presentation is based on work with many individuals including Eric Laber, Dan Lizotte, John Rush, Scott Stroup, Joelle Pineau and Susan Shortreed. Email address: samurphy@umich.edu Slides with notes at: http://www.stat.lsa.umich.edu/~samurphy/ Click on seminars > health science seminars

31. Voting • (exploratory analysis) • Use bootstrap samples to estimate percentage of the time that treatment A1=1 is favored: • Natural approach will not work, e.g. • is not necessarily consistent. • We use an adaptive bootstrap 31

32. Voting • (exploratory analysis) • Use an “adaptive” bootstrap method to estimate percentage of the time that treatment A1=1 is favored: • Adaptive bootstrap estimator: 32

33. Treatment of Schizophrenia Myopic action: Offer patients a treatment that reduces schizophrenia symptoms for as many people as possible. The result: Some patients are not helped and/or experience abnormal movements of the voluntary muscles (TDs). The class of subsequent medications is greatly reduced. The mistake: We should have taken into account the variety of treatments available to those for whom the first treatment is ineffective. The message: Use an initial medication that may not have as large a success rate but that will be less likely to cause TDs. 33

34. Treatment of Opioid Dependence Myopic action: Choose an intensive multi-component treatment (methadone + counseling + behavioral contingencies) that immediately reduces opioid use for as many people as possible. The result: Behavioral contingencies are burdensome/expensive to implement and many people may not need the contingencies to improve. The mistake: We should allow the patient to exhibit poor adherence prior to implementing the behavioral contingencies. The message: Use an initial treatment that may not have as large an immediate success rate but will allow us to ascertain whether behavioral contingencies are required. 34

35. Example of an Adaptive Treatment Strategy Treatment of depression. Goal is to achieve and maintain remission. Provide Citalopram for up to 12 weeks gradually increasing dose as required. If, there is no remission yet either the maximum dose has been provided for two weeks, or 12 weeks have occurred, then if there has been a 50% improvement in symptoms, augment with Mirtazapine. else switch treatment to Bupropion. Else (remission is achieved) maintain on Citalopram and provide web-based disease management. 35