Interim Analyses of Clinical Trials

1. Interim Analyses ofClinical Trials

2. Outline Background and how DSMBs arose and function Group sequential methods Examples

3. Suggested Reading Ellenberg S, Fleming TR, DeMets DL Data Monitoring Committees in Clinical Trials. A Practical Perspective John Wiley & Sons, LTD, 2002

4. Research on Human Subjects and Data Monitoring Committees Nuremburg Code (1949) Declaration of Helsinki (1964) Belmont Report (1979) Greenberg Report (1967) FDA Guidance on DMCs (2006)

5. Structure for Cooperative Studies (Greenberg Report)

6. Monitoring Committee Acronyms PAB = Policy advisory board DSMB = Data and Safety Monitoring Board DMC = Data Monitoring Committee ESMB = Efficacy and safety monitoring board OSMB = Observational study monitoring board

7. Responsibilities Steering Committee Study design Patient recruitment and follow-up Data collection Quality assurance Study reports DMC or DSMB Safety of patients Protection of integrity of study Review of blinded data on safety and efficacy of treatments Review of trial conduct, amendments and external data

8. Starting Point A randomized study Uncertainty with respect to the relative safety and benefits of treatments to be compared Protocol to address the scientific question Informed consent with explicit or implicit mention of interim monitoring Plan for timely data collection and safety monitoring

9. In the End Want� Confidence in the answer A report in which you can specify: - Number of interim analyses - Reason for stopping, if early - Monitoring guidelines used - DSMB membership

10. Interim AnalysesRationale Safety is best assured by comparing the rate of adverse events with a control group Studies should not stop before there is a definitive answer and should not continue longer than necessary to obtain one Regular assessment of the relevance of the question Regular assessment of whether the trial will address the question posed

11. Reasons for Early Terminationof Clinical Trials Based on accumulated data from the trial: Unequivocal evidence of treatment benefit or harm Unexpected, unacceptable side effects No emerging trends and no reasonable chance of demonstrating benefit Based on overall progress of the trial: Failure to include enough patients at a sufficient rate Lack of compliance in a large number of patients Poor follow-up Poor data quality

12. Today All NIH sponsored clinical trials are required to have a data monitoring plan NIH-sponsored trials with clinical endpoints have a DSMB Many industry sponsored studies have a DSMB The FDA has prepared a guidance document that will likely result in many more industry-sponsored trials with a DSMB There is considerable variation in operating procedures for DSMBs

13. When is an Independent DSMB Needed Early phase studies Monitoring usually at local level; independent DMC not usually needed. Phase III & IV studies with morbidity/mortality outcomes; pivotal phase III trials Frail populations, e.g., children, elderly Trial with substantial uncertainty about safety, e.g., gene therapy

14. DSMBComposition: Multidisciplinary Clinical experts in the subject matter area Biostatisticians with expertise in clinical trials and preferably in the subject matter area Others depending on the nature of the study, e.g., ethicist, pharmacologist, patient advocate Senior investigators without significant conflicts of interest

15. Independence of DSMB: Voting members should not be part of the investigative team or work for the sponsor There should be a clear �need to know� policy for non-DSMB members, e.g., the statistician preparing interim summaries needs to know and may be an employee of the sponsor or member of the investigative team Members should state potential conflicts

17. Two-Hat Model: In-house statistician who does everything Same person(s) participates in protocol development, interim analyses and writing papers (Steering Committee) Model used by many NIH-sponsored trials May be only option for small company Important to keep a poker face

18. Protocol amendments may be influenced by knowledge of interim data Difficult to avoid arousing suspicion ��even unintentional non-verbal communication (e.g., nervousness, smiling) may reveal some of that knowledge.� Draft Guidance Document: On the Establishment and Operation of Clinical Trial Data Monitoring Committee, FDA, November 2001. Disadvantage of Two-Hat Model

19. One Hat Models Two statisticians, one who works with the investigators on the protocol and the other who carries out interim analyses 1 in-house and 1 external 2 in-house 2 external Are there enough statisticians?

20. One Hat (2 Statisticians) vs. Two Hats (1 Statistician) Leakage less likely More �objective� DSMB reports Greater external credibility Greater knowledge of study/field More thoughtful DSMB reports Cheaper Professionally more rewarding This is where you add text of points that you want to make on this particular slide.This is where you add text of points that you want to make on this particular slide.

21. Rules of the Game Charter with lines of reporting stated Role in review of protocol and amendments Open and closed sessions for DSMB meeting (open � closed � executive or closed � open - closed) For early stopping for efficacy, there should be proof beyond a reasonable doubt Guidelines should not be too restrictive, e.g., number of interim analyses

22. Rules of the Game (cont.) Ability to schedule meetings and request additional analyses deemed necessary Non-blind monitoring Minutes of open and closed session Specify minutes-taker

23. Outline of Charter Membership and voting rules Meeting format and frequency Reporting responsibilities Stopping guidelines Minutes Procedures to ensure confidentiality (treatment codes, disposition of reports) Conflicts of interest

24. DSMB Meeting Format Open Session Progress report using open data Sponsor, e.g., NIH, Executive Committee, Protocol Chairs, DSMB and unblinded statisticians Closed Session Blinded data by treatment group DSMB and unblinded statisticans only Executive Session (DSMB only) Debriefing Session DSMB, Sponsor, Executive Committee, Protocol Chairs, and unblinded statisticians

25. DSMB Confidentiality Interim data reviewed by the DSMB must remain confidential Members must not share interim data with anyone outside DSMB Leaks can affect Patient recruitment Protocol compliance Outcome assessment Trial integrity and support

26. Sponsor Executive committee Steering committee DSMB Decision Making Based on Interim Analyses

27. Recommendation No one other than statistician who prepares the report is present during review of treatment comparisons It is generally inappropriate for the sponsor (NIH or industry) or the FDA to be present or to be given interim treatment summaries The inclusion of others requires justification � �need to know�

28. Open Statistical Report � Findings for All Treatment Groups Combined Outline of the study design Major protocol changes Study accrual by month and by institution Eligibility violations Baseline characteristics Days between randomization and initiation of treatment Adherence Attendance at scheduled visits Reporting delays for key events Length of follow-up data available Assessment of sample size assumptions

29. DSMB Reporting Inclusion of irrelevant material interferes with the primary job of the DSMB and makes members testy Highlight most important results

30. Controversial Issues About Open Data? Risk factor or marker changes known to be on the pathway between treatment and clinical endpoint Adherence to individual treatment assignments Total number of events (all treatment groups combined) When should there be a middle ground, e.g., safety data?

31. Closed Statistical Report Detailed statistical commentary explaining issues raised by closed report figures and tables (by coded treatment group, with codes given to DSMB members) Summary of closed report data presented at prior DSMB meetings Repeat of the open report information, in greater detail by treatment group Analyses of primary and secondary efficacy endpoints Analyses of adverse events and overall safety data Treatment discontinuations and crossovers Listing and summary of major events, e.g., deaths and serious adverse events (SAEs)

32. A Typical Safety Section of DMC Report � Important Not to Miss Forest for Trees! 100-300 pages Line listings Adverse event frequencies All events Emerging events Severe events Events leading to discontinuation of treatment Laboratory data Frequency of abnormal findings Average changes over time

33. Typical Questions/Problems Are you counting events or patients? What time period do %�s pertain to? Did you collect events after patients discontinue treatment? Which terms do I look at to determine how many patients experienced a heart attack? Are patients being counted more than once - Single event and more than one name? Multiple events occurring at different points of follow-up? Suppose I just want to know how many patients in each arm have experienced severe depression?

34. DMC Report � Mood Disturbances

36. Pitfalls of Lumping versus Splitting:Myocardial Infarction

37. Should Level of Severity be Considered in the Collection of Safety Data? Yes Lower grade events in earlier phase studies More severe events only in large outcome studies

38. Should Level of Perceived Relationship to Study Treatment be Considered in the Collection of Safety Data?

39. How Can the Data be Summarized to Facilitate DMC Reviews? A priori agreement on a data analysis plan, that includes: A hierarchy of events and associated composites: Death Death or serious AE Death or serious AE or treatment D/C due to AE Death, serious AE, treatment D/C due to AE, or grade 4 event Categories of events of interest (granularity) Interim efficacy as well as safety for assessing risk/benefit For key events, a history of previous interim analyses

40. Summary -1 A DSMB can be most effective in its role of protecting the interests of patients if it is independent of the sponsor and trial investigators � peer review works! Operating procedures should be agreed upon in advance An informed statistician who performs interim analyses is important To carry out interim analyses data must be collected in a timely way Reports should focus on comparisons of clinical outcomes and their validity

41. Summary - 2 Collect less data and do it well. Focus on complete collection of severe events irrespective of perceived relationship to treatment Collect these events over the entire duration of the study Develop a data analysis and reporting plan for adverse events before the study. Prepare reports in a manner to allow DMC members to easily assess emerging signals. Include interim efficacy analyses with safety analyses in order to assess risk/benefit.