Study Designs for Acute Otitis Media: What can each design tell us?

1. Study Designs for Acute Otitis Media: What can each design tell us? C. George Rochester, Ph.D. Anti-Infective Advisory Committee Meeting, July 11, 2002 Mathematical Statistician, Division of Biometrics III RochesterG@cder.fda.gov

2. Outline • The role of tympanocentesis (TAP) in AOM trials • Advantages and disadvantages of each design • Superiority design: placebo-controlled (PCT) • Non-inferiority (NI) Anti-Infective AC Meeting 7/11/02

3. Population Studied • AOM represents a spectrum of illness • Proof of efficacy include • Clinical proof • Microbiologic proof • Cautions: • Guard against post-hoc subset analyses as proof • Extrapolation to populations not directly studied Anti-Infective AC Meeting 7/11/02

4. Current state of affairs • Clinical only, comparative, non-inferiority study: new versus standard • Reference to Non-inferiority design (R. Dagan, Nov 2001, AIAC): “…most of the acute otitis media trials with clinical outcome as currently conducted are virtually guaranteed to show no differences between agents, dosing, or duration of treatment .” • Baseline bacteriology and clinical outcome - often non-comparative Anti-Infective AC Meeting 7/11/02

5. Why do some trials fail to detect differences among treatments? • Differences among arms may truly not exist • “Noise” usually makes arms converge • Sources of noise in AOM studies • Enrollment of subjects without bacterial infection at baseline (viral infection), loose case definition • Spontaneous resolution even with bacterial infection and varies with different organisms • Determination of treatment response includes subjective components which may be subject to inter-rater variability • Strategies for handling noise: • Placebo controlled trials: if difference is observed a treatment benefit is established • Non-Inferiority trial: Baseline TAP reduce noise in diagnosis; Repeat TAPS reduce noise in outcome assessment Anti-Infective AC Meeting 7/11/02

6. Should TAPS be performed? • Placebo-controlled trials • Clear evidence of clinical benefit • TAPS add efficiency • Baseline TAP • Improves NI design where “noise” may lead to false proof of efficacy • Follow-up TAP • Bacteriological outcome is more objective • The optimal time and number of TAPS to perform may need further research • On-therapy (e.g. Day 3-5, or 4-6) • TAP all failures Anti-Infective AC Meeting 7/11/02

7. Single TAP at baseline • Bacteriologic diagnosis and clinical outcome assessment • Baseline TAP ensures that patients in the primary analysis have baseline pathogens • Better than no TAPS but bacteriologic outcome is presumptive • In practice failures do not usually get follow-up TAPS regardless of protocol specification • NI with baseline TAP • May allow a wider non-inferiority margin (smaller sample size) than NI without baseline TAP Anti-Infective AC Meeting 7/11/02

8. Repeat TAPS • Provides objective bacteriologic outcome • Blinding not as critical for bacteriologic endpoint but is essential to reduce bias if the clinical outcome is the ultimate goal • Study is successful if efficacy is shown at microbiologic and clinical assessment time points Anti-Infective AC Meeting 7/11/02

9. Fundamental question regarding the utility of a microbiologic endpoint • Bacteriologic endpoint is a “surrogate” and correlation with clinical endpoint may be less than satisfactory given current data • Much uncertainty about the microbiologic endpoint remains Anti-Infective AC Meeting 7/11/02

10. Agency for Health Research Quality Evidence Report (2001): Management of Acute Otitis Media “There is still a need to adequately address the role of antibiotics in the initial treatment of AOM in children compared to placebo or observational treatment especially in terms of various influencing factors such as age and otitis-prone status. Close monitoring of patients in these studies with a priori plans for appropriate intervention should allay any concerns about suppurativecomplications and should also be a focus of research.” Anti-Infective AC Meeting 7/11/02

11. Randomized, double-blind, placebo-controlled trials • PCT is the GOLD STANDARD • Efficient and easy to interpret: direct evidence • Three arm trials - New drug, standard, placebo • Blinding (double): reduce experimental bias during study conduct • Randomization: ensure balance in treatment groups for known and unknown factors • Placebo: direct estimate of treatment benefit • Provides scientific foundation on which to plan future active-controlled trials Anti-Infective AC Meeting 7/11/02

12. Advantages and disadvantages of placebo-controlled trials Anti-Infective AC Meeting 7/11/02

13. Non-inferiority trials • Compare new drug against a “standard” • Estimate of benefit of new drug depends intricately upon knowing the benefit of the standard over placebo • Efficacy is indirect (relative) and relies on choice of an appropriate non-inferiority margin (delta) • Efficacy is demonstrated if we are certain that benefit of active control over placebo on primary endpoint is greater than the non-inferiority marginin the population studied • Choice of non-inferiority margin also depends on microbiologic rigor Anti-Infective AC Meeting 7/11/02

14. Non-inferiority design without TAPS Anti-Infective AC Meeting 7/11/02

15. Non-inferiority design with baseline TAP Anti-Infective AC Meeting 7/11/02

16. Advantages Disadvantages • Assess both endpoints • Delta micro (DM) • Delta clinical (DC) • Microbiologic endpoint is a surrogate for the ultimate clinical outcome and may not be highly predictive • There may be ethical and practical limitations • Micro endpoint objective Non-inferiority design with repeat TAPS Anti-Infective AC Meeting 7/11/02

17. Which design to use? • To demonstrate absolute efficacy: PCT • To demonstrate absolute and relative efficacy: PCT with 3 arms • If the magnitude of the advantage of the active control over placebo is known for the primary endpoint: NI design (ICH E-10) • NI margin must be less than this advantage • Be conservative if historical data is poor • Be conservative if prior information is not relevant to current population/endpoint Anti-Infective AC Meeting 7/11/02

18. What does each design tell us? Anti-Infective AC Meeting 7/11/02

19. Summary • TAPS improve the efficiency of AOM trials • Repeat TAPS provide objective microbiologic information • Placebo-controlled trials • Efficient and easy to interpret • NI design should be considered when • The benefit of the standard over placebo is known • Microbiologic rigor can improve the quality of these trials • Is the microbiologic or the clinical endpoint more desirable to patients? Anti-Infective AC Meeting 7/11/02

20. Credits Erica Brittain, Ph.D. Daphne Lin, Ph.D. Ruthanna Davi, M.S. Karen Higgins, Sc.D. C. George Rochester, Ph.D. Clinical reviewing divisions: • Division of Anti-Infective Drug Products- HFD-520 • Division of Special Pathogen & Immunologic Drug Products- HFD-590 Anti-Infective AC Meeting 7/11/02