Efficient: The correct answer about a treatment’s clinical

1. Can we improve the efficiency of definitive clinical trials of treatments to protect beta-cells in persons with or at risk for T1D? Efficient: The correct answer about a treatment’s clinical benefit with as few subjects as possible as quickly as possible. Definitive: Yields results that change clinical practice. We’re focusing today on design and analysis options. What about the effect size?

2. I’ll talk about: •The effect size in general terms. • The clinical justification for smaller definitive trials that are only powered to detect larger effects. • The disadvantages of smaller trials. • Focus will mostly be on prevention trials.

3. The Effect Size Sample size estimates for T1D prevention trials:  Risk for diabetes in control group  Risk for diabetes in experimental group  Risk for a false-positive conclusion  Risk for a false-negative conclusion The effect size is the difference in risks for diabetes. Larger effects yield smaller sample sizes.

4. Biological Versus Clinical Effects Biological effects: • Concerned with variables bearing on inferences re: disease pathogenesis, mechanism of action, pharmacokinetics/dynamics, etc. • Primary focus in Phase 1/2 trials (but can be important in Phase 3 trials). • Not always clinically important, at least in the short-term. Clinical effects: • Concerned with variables that matter to patients – “live better, live longer”. • Primary focus in Phase 3 (definitive) trials.

5. Choosing an Effect Size Operational Definition: •The smallest difference that would make patients and clinicians want to use the new treatment or theMinimal Clinically Important Difference (MCID). • Other terms: MID, CMD, MDD. Other Inputs: • Prevailing view that highly effective treatments are uncommon, eg., RRR in diabetes rates that exceed 50%. • Results of Phase 2 trials and/or observational studies in humans. • Practical considerations: “How many subjects can I get?” instead of “How many subjects do I need?”

6. Choosing an Effect Size • The MCID approach is not ideal. Nevertheless, it focuses pre-trial thinking on clinical issues. •Larger MCIDs yield smaller sample sizes. • It’s therefore important that TrialNet think carefully about the MCID and, if the clinical argument is strong, consider accepting larger over smaller MCIDs.

7. The Case for Choosing Large MCIDs in TID Prevention Trials Start at the end and work backwards: • TrialNet completes a prevention trial that is “positive”, ie., reduced risk for diabetes over the trial’s duration ( 5 years). • Translating the results to clinical practice will need to deal with real and perceived concerns among clinicians and patients to whom the results will apply. • What are those concerns?

8. Concern Number 1. For most treatments, the risk for adverse effects in the long-term (eg., 5 – 10 years) are unclear, esp. in kids. Otherwise well persons will receive a potentially toxic treatment for a problem (future diabetes) that, in some cases, they aren’t going to develop in the first place.

9. Concern Number 2. The efficacy-effectiveness “dilemma”: The benefits and risks seen in a Phase 3 trial under ideal (efficacy) conditions will be, respectively, over- and under- estimated in routine practice (effectiveness). This applies to most clinical trials -- it will receive greater emphasis in T1D prevention trials.

10. Concern Number 3. Treatment for T1D has improved over the last 2 decades, and is likely to continue to improve over the next decade. The new treatment option of beta-cell protection, in whatever form it ultimately takes, will not be used in routine practice if there are doubts it’s not better relative to the existing standard of care.

11. Predictions re: impact of these evidence-based concerns: 1. Exact level of resistance to adapt any single treatment into practice can’t be predicted. However, in some cases it will prevent translation to practice. 2. All other things being equal, prevention trials showing large effects will have a greater chance of being accepted over trials showing small effects. (The MCID is bigger than we think?)

12. Numbers Needed to Treat (NNT) and Numbers Needed to Screen (NNS) to prevent 10 cases of diabetes/5 years according to effect size. • Effect Size • Diabetes rates over 5 years: • Controls 50% 50% 50% 50% • Experimental subjects 33% 25% 17% 12% • Relative risk reduction 33% 50% 67% 75% • Total N randomized 256 110 56 40 • NNT to prevent 10 cases/5yrs: 60 40 30 26 • Number who develop DM despite Rx 20 10 5 3 • Number who will not develop DM but 30 20 15 13 • who still receive the Rx • NNS to prevent 10 cases/5yrs 13,620 9,080 6,810 5,900

13. Small trials have important limitations that have to be accepted or accomodated for.

14. We will miss treatments that are not highly effective, because most treatments won’t exert large effects. • Negative impact on Study Group and third party payers as several negative (“no effect”) trials accrue. • Modestly effective treatments may prove to be clinically important. • Lost opportunity to combine 2 or more modestly effective Rxs in subsequent trials for additive/synergistic effects. • Miss biological inferences based on the Rx’s mechanism of action relative to mechanisms of beta-cell destruction.

15. Responses: “We will miss real effects in smaller trials, but I don’t care. That’s because I think treatment effects smaller than what we’re able to detect in a small trial are not clinically or biologically important.” The “silver bullet” gamble may not be as risky as we think: • There are many promising-looking treatments at hand • There are precedents, egs., The CERT of Cyclosporine The French anti-CD3?

16. Greater potential impact on the trial’s conclusion because of imbalances in baseline variables associated with the outcome. Response: • There are ways to deal with this, eg. stratification. • Our challenge – agreeing on the variables that are important enough to need attention: • Prevention trials: good shape. • Intervention trials: not quite as clear?

17. Reduced power to detect adverse effects, especially uncommon or rare serious adverse effects. Response: • Randomized trials have never been a good tool to determine risks for rare events. • Followup of all randomized subjects over an extended time (~ 5 to 10 years) beyond completion of trial’s initial efficacy phase is needed no matter how big the trial was. (To me, this is the single most important reason against us doing small trials.)

18. Summing Up TrialNet should consider shifting attention toward larger effect sizes (MCIDs), eg., RRR ~ 67% in diabetes rates in T1D prevention trials, at least for the general case. The final choice for the MCID must still be adjusted according to information that is specific to the test therapy. Smaller trials have important disadvantages that we have to either accept or address during trial planning and post-trial followup.

19. N/group = (Z + Z) 2 [(Pr C)(1 – Pr C) + (Pr E) (1 – Pr E)] (Pr C – Pr E) 2 Z= 1.96 (5% 2-tails), Z= 0.84 (80% power) Pr C = diabetes rate/5 yrs in controls Pr E = diabetes rate/5 yrs in experimental subjects