ISSUES THAT PLAGUE NON-INFERIORITY TRIALSPAST AND FUTURE RALPH B. D’AGOSTINO, SR. BOSTON UNIVERSITY HARVARD CLINICAL RESEARCH INSTITUTE
OBJECTIVES • REVIEW ISSUES: PAST, PRESENT AND FUTURE IN NON-INFERIORITY (NI) STUDIES • PRESENT/ DISCUSS EXAMPLES • MAKE SOME COMMENTS FOR IMPROVEMENTS • PRESENT A PERSONAL VIEW
OUTLINE • Early Objectives and Issues • Approaches to Non-inferiority Trials • Examples (Here are some Problems) • Non-Inferiority AND/OR Superiority • All is Non-Inferiority • Intent-to-Treat vs. Per Protocol • New Major Issues
EARLY OBJECTIVES AND ISSUES: EQUIVALENCY • American Dental Association (ADA 1980s) • CREST equivalent to COLGATE? • Ho: A-B>= 10% or A-B<= 10% • What does the 10% mean? • DFMS or DFMT for 2 years, 3 years? • Study done on differences and ratio used as descriptive measure of “effect” • 5.0 vs 5.4 becomes (5.4-5.0)/5.0 = .4/5.= 8%
EARLY OBJECTIVES • M = 10% CAME FROM NOWHERE, BUT WE KNEW WHAT IT WAS, That is, 10% • TREATMENT DIFFERENCES CONCERNED DIFFERENCES (RATIOS) BETWEEN ACTIVE TREATMENTS • WE WERE LOST BUT WE BELIEVED WE HAD A “SENSE” ABOUT IT
APPROACHES TO NI TESTS • MUST DO BETTER THAN PLACEBO • But you cannot use a Placebo (P) • Putative Placebo Approach • Test Treatment (T) vs Positive Control (C) directly with given Margin M (Assay Sensitivity approach)
APPROACH 1 (Putative Placebo)Stellar Example from the Past • CAPRIE Study. Hasselblad and Kong (2001) present this as their major example for using meta-analyses for dealing with estimating assay sensitivity (T vs. P) • Want T vs. C, C vs. P, T vs. P
CAPRIE STUDY (cont) • Can we obtain effect of Clopidogrel vs. Aspirin • Yes, if we can locate Asprin vs. Placebo • Do we believe what we get?
For Aspirin vs. PlaceboAntiplatelet Trialists’ Collaboration Meta-Analysis • Meta-analysis of all published and unpublished unconfounded randomized trials available March 1990 • Trials identified by literature search, trial registry and inquiry of investigators and pharmaceutical manufacturers • Clear definitions of endpoints • Well defined statistical methodology
APPROACH • T vs. C (from Caprie trial) • C vs. P (from Meta-analysis) • Obtain T vs. P (from multiplication) • (T/C) (C/P) = (T/P)
Clopidogrel Vs. Synthetic Placebo Control Odds Ratios and 95% Confidence IntervalsOverall Patient Population CAPRIE: Clopidogrel Vs. Aspirin Meta-Analysis: Aspirin Vs. Placebo Estimated: Clopidogrel Vs. Placebo Endpoint All Strokes, MIs, or Vascular Deaths p < 0.000001 All Strokes, MIs or Death from p < 0.000001 Any Cause Vascular p < 0.0016 Deaths All Cause p < 0.0045 Deaths 0.4 0.6 0.8 1.0 1.2 1.4 1.6 First Drug Better Second Drug Better
Meta-analysis studies contain very old studies (only up to 1990), many prior to all of the elaborate medical interventions (procedures) now routinely provided • Many of the studies did not have MI, IS or vascular death as their outcomes (the meta-analysis went back to original investigators who in turn, had to generate data). Ever try to get data on something you did not collect? • None of the studies used for Clopidogrel with aspirin comparison had PAD as an entry criteria (PAD represented 1/3 of Clopidogrel Study)
EFFECT SIZE: Relative Risk Reduction by Qualifying Condition (ASA vs Clopidogrel) IS n = 6431 MI n = 6302 PAD n = 6452 Total n =19185 30 20 10 0 10 20 Clopidogrel Better Aspirin Better
Problems With Historical Controls • Biases • Time Biases • Change in recognition or diagnosis of disease • Changing disease process • Change in usual therapy (Myocardial Infarctions MI, Dx, Tx) • Selection Biases • Patients/Health care systems • Are we really seeing the same patients in historical studies as those in active control trial?
Problems With Meta-AnalysesSo What Is Sponsor to DO? If we plan to use placebo controlled trials, what should we require of the historical placebo trials? • Same Disease/Conditions? • Same Population • Same Dose and Administration Levels of Active Control C? • Same Outcomes? • Combine “All” or “Some (good)” Placebo Controlled Studies
Still Other Problems With Meta-Analyses • What if previous studies had multiple arms? How to put correctly into meta-analysis? • What if none of the individual studies achieved significance? • What are we to believe from meta-analyses? • Do we believe the p-levels of the meta-analysis? (I do not think we should.)
APPROACH 2NON-INFERIORITY STUDIESACTIVE CONTROL STUDIES NON-INFERIORITY TEST H0: T-C >= M vs. H1: T-C < M (Say data are event rates) T is new treatment C is positive control M IS NON-INFERIORITY MARGIN
NON-INFERIORITY STUDIES APPROACH 2 • SELECT A VALUE OF M THAT MAKES SENSE • WANT ASSURANCE THAT ASSAY SENSITIVITY IS PRESENT (Placebo is working) • WANT CONSISTENCY WITH PAST
NON-INFERIORITY STUDIES Statistical Approach • Need Active Control C vs. Placebo P data from Historical data (C vs. P) • Need to test effectiveness of T vs. C • Need estimate of fraction of C-P preserved by T (e.g., (T-P)/(C-P) = M) M=0.5 (C-P) METHODS EXIST THAT ALLOW TEST TO BUILD IN NEW AND HISTORICAL DATA (STATISTICS IN MEDICINE, 2002)
WHAT IS NEEDED FOR 2 • CONFIDENCE INTERVAL IS OFTEN USED. WANT M=1.11 (SAY) OUTSIDE UPPER LIMIT OF CONFIDENCE INTERVAL (M is relative risk) • FDA ODAC 8/04 (non-small cell lung cancer) 1.0 1.11= M
SOME REALITIES • Sounds nice • What happens
Anti-infective Product No placebo data • Historical data is not Placebo, but C • VRE (vancomycin resistant enterococcal) High dose Low dose • MITT 60.0 % (N=65) vs. 46.2 % (N=52) • Bacteremic 55.6 (N=18) vs. 25.0 (N=16) • What is M? One trial OK? Any superiority?
ANOTHER EXAMPLERespiratory Distress • Respiratory Distress Syndrome in Premature Infants • Treatments • New Drug • Comparator • Outcome • Survival at 28 Day
Respiratory Distress (cont) • Survanta versus Sham (two studies one positive, other negative) All Cause mortality • Study 1: 8% vs. 23% Study2: 17% vs. 14% • What is M? .23-.08? .180-.125?
CONSISTENCY Example Control rate different from historical • Historical Data says C=0.5 and P=0.6 • Want T<=0.55 • P-C=0.10, M=0.5(0.10) = 0.05 • (T-C)/C = 0.05/0.50 = 10% • Data is C=0.30 and T=0.33, T-C=0.03 • (T-C)/C = 0.03/0.30 = 10% • IS STUDY A SUCCESS? USE RATIOS?
ANSWER TO CONSISTENCY • There was consistency • Differences related to birth weight
Non-Inferiority and Inferiority at the same time • Sponsor falls apart 0 M
Non-Inferiority and Superiority • Sponsor jumps for joy (Sequential test) 0 M
Switching trial design (Cardiac Stent Trials) • (1) New drug coated stents, we can do non-inferiority study with margin set (15%) • (2) We can do superiority study with non-coated stent as control • With first option we have to worry about evaluating Ms, Effect size and CREEP • With superiority trial “clean” results
Respiratory Distress • Compare new surfaxin to another “not so great” one, but still used in practice
Switching from Superiority to non-Inferiority • HOW CAN WE SWITCH FROM A SUPERIORITY TEST TO NON-INFERIORITY ? • This is a question thrown at me constantly
Assessing Efficacy Non-Inferiority and Safety Superiority • Carotid artery Magnetic Resonance Imaging agent • Imaging Agents • Agent N (New) Agent C (Comparator) • Non-inferiority” Outcome • Endpoint: agent’s ability to classify correctly patients with > 25% stenosis (sensitivity) • Sensitivity of Comparator is .80 or 80% • Non-inferiority margin M set to 0.10
Assessing Efficacy Non-Inferiority and Safety Superiority (Cont’d) • There is a specific adverse event that is hypothesized to occur less often with New than with Comparator • Do we want to make the specific adverse event rate an additional primary endpoint? WHY NOT?
Non US STUDIES • Forced off shore (ethical and other reasons)
The BLOB EFFECT • Everything is suddenly Non-Inferiority
ALLHAT STUDY • COMPARISON OF ANTI-HYPERTENSIVE MEDICATIONS (MULTIPLE ARMS) • NOT A NON-INFERIORITY STUDY
Safety Studies • Safety studies have become carefully designed and executed studies • Should they be non-inferiority studies?
SAFETY STUDIES (PHASE 4) HISTORICAL APPROACH: NEW RATE > OLD H01: T-C <= 0 vs. H11: T-C > 0 H02: RR=T/C <= 1 vs. H12: RR=T/C > 1 STUDY POWERED TO REJECT T/C >1.5 (SAY) SHIFT IS TO MAKING THESE NON-INFERIORITY STUDIES • H0: T-C >= M vs. H1: T-C < M H0: RR=T/C >= M vs. H1: RR=T/C < M
Safety Studies • OLD • NEW 1 M
SAFETY STUDY TO NON-INFERIORITY STUDY(QT LONGATION) • Safety issue: drug may cause QT problem • Ho: A/B = 1.0 vs H1: R = A/B > 1.0 • Study powered for R > 1.0 • When interest in risk fades can we suddenly say this should be a non-inferiority study? • Ho:R >= 1.5 vs. H1:R < 1.5 was not original objective • If we do not reject Ho is that enough?
Form of Interest and Sample Size • Ho: p1-p2 >= M • Ho: p1-p2>=Rp2 • Ho: p1/p2 >= R • Best Choice does depend on p2 (control rates)
Intent-to-Treat vs. Per-Protocol • In superiority trials, the primary analysis is often on intent-to-treat (ITT) population • Per Protocol (PP) “bigger” differences of treatments • In non-inferiority should we use PP?
Intent-to-Treat vs. Per-Protocol (Cont’d) • PP as primary not always accepted • “the ITT analysis is as important as the PP analysis” • “need to reconcile differences between ITT and PP analysis” • Perform “sensitivity” analyses. Results should be similar in both populations (ROBUSTNESS). • The Committee on Proprietary Medicinal Products draft Points to Consider: “…similar conclusions from both the ITT and PP are required in a noninferiority trial”.
We ask sponsor to do both (ITT and PP) and expect to achiev the sam significant result on both. • What is the true alpha associated with this?
NEW MAJOR ISSUES • Missing Data • Noncompliance • Interim Analysis • OUR USUAL LOGIC INCREASES CHANCE OF ACCEPTANCE OF non-inferiority
MORE NEW ISSUES • Multiple endpoints • Multiple groups • Repeated Measures
WHERE ARE WE? • NON-INFERIORITY TRIALS HAVE MADE A BIG IMPACT • They have brought many new problems and challenges with them