Intervention Trials Principles of study design

1. Intervention TrialsPrinciples of study design James B. Spies M.D.MPH Professor of Radiology Georgetown University School of Medicine Washington D. C.

2. Categories of StudiesUsers Guide to Medical Literature • Studies answer one of 4 types of questions: • Therapy (efficacy) • Randomized trials, other comparative studies • Harm • Cohort or case control studies • Diagnosis • Diagnostic studies comparing new study to reference standard • Prognosis • Cohort studies

3. Efficacy Trials • What type of study design? • Case reports • Case series (+/- historic controls) • Retrospective or prospective • Prospective comparative studies w/o randomization. • Randomized comparative studies

4. Non-randomized designCase Series- Weaknesses • Selection bias • The investigator controls recruitment and may enroll those who are better candidates for the intervention compared to the typical patient. • Does not allow for control of any known or unknown confounders in subjects. • Often done retrospectively, in which incomplete medical records are abstracted. • Does not allow for clear cut definition and recording of baseline characteristics, technical aspects of therapy, or outcome.

5. Non-randomized designCase Series • No controls provides no means of assessing study impact. • Before and after within subject control does not allow assessment of placebo effect (if not blinded) and any improvement that may occur without intervention. • Historical controls subject to even greater selection bias • Research staff collecting data are typically not blinded and significant assessment bias may be introduced.

6. Non-randomized designCase Series • Investigators not blinded • Researchers may have vested financial interest in positive outcome. • Investigators may be on company speaker’s bureaus, be consultants, or on medical advisory board. • This is why investigators required to report conflicts at presentations and in publications. Red Flag. • May have a personal vested interest • The study is usually done with the intent and the hope that the new therapy is an improvement over current therapy. • New therapies become associated with the investigator, potentially leading to a bias supporting positive outcomes.

7. Non-randomized designNon-randomized comparative prospective studies • Improved opportunity for parallel assessment of subjects and contemporaneous controls. • Data definitions and collection can be more complete • Can compare subjects and controls for similarities. • Selection bias still key problem. • Asessors, patients usually not blinded.

8. Randomization • Controls selection bias • Primary benefit • Best means of assigning the known and unknown potential confounders in each arm. • Factors that can impact outcome are randomly distributed. • Even when unknown, confounders are controlled for. • Allows for application of statistical models.

9. Randomization methods • Coin toss: subject to bias • Random number tables or computer program • Computer-generated less subject to bias. • Should be done by a third party • For procedure trials where physician not blinded, assignment either done in central call-in center or put in opaque envelopes and revealed to investigator at last possible time. • Prevents investigator from gaming assignment.

10. Randomization schemes • Simple • Each assignment is independent of the previous and the next. • Blocked • Blocks of assignment of varying lengths. • Blocks of 2, 4, 6, 8 mingled. • Provides for balanced assignment over a fixed number of participants. • May be used to avoid very uneven numbers of patients in each treatment group. Most important in small studies. • Length of blocks varied and secret- prevents guessing as to which assignment is next.

11. Randomization schemes • Stratification • Dividing the assignment groups by some important clinical parameter. • Race, age group, gender, study center, disease severity, etc. • Assures equal numbers among important subgroups. • Usually only 2 groups or 3 groups stratified. • Can combine stratification and blocking • Will have different blocking groups for each subgroup.

12. Randomization schemes • May use unequal allocation of subjects • 1:2 or 1:3 • Helpful when one treatment inherently more attractive to potential subjects • Applicable when a well-studied standard therapy is used as control. • There may less need for data regarding safety on standard therapy than the new therapy, so fewer standard therapy patients may be needed.

13. Blinding • Extremely important to blind or mask as many of the key participants as possible as to assignment. • Physicians, assessors, nurses, patients. • All can contribute biases to the outcome assessment. • Internal validity depends on removing selection bias (randomization) and removing assessment bias (blinding).

14. Therapeutic study designs • Parallel • Two groups randomized, treated and followed as two groups. • Crossover • Two groups randomized, each starts one treatment and then at a designated time point, they crossover and receive the other treatment. • May need a washout period prior to starting drug or therapy 2, to allow for effects of drug one to wear off.

15. Choice of control • Placebo • Relatively easy to implement in drug trials. • Easy standard for approval or acceptance of a new drug. • May not be useful if other effective similar drugs are already approved. • Most recent standards for ethical conduct of trials bar use of placebos

16. Choice of control • Active controls • Most ethical approach is a current standard of care (if one is available). • Raises problems for analysis • Should the new treatment be better than the old? • May not be feasible if gold standard is highly effective. • It may be new therapy is less invasive, less expensive, or otherwise advantageous, but is not better. • May not be necessary to show superiority

17. Trial Design • Superiority, Equivalence, or Non-inferiority • Superiority ideal if a significant improvement over conventional therapy anticipated. • Equivalence difficult to show • Need to have minimal difference between groups, which requires large number of subjects. • Non-inferiority trials- the new therapy is not worse than current. • Allows investigators to decide the margin that is acceptable for non-inferiority. • Could be a margin of 10 or even 20 percent less effective, but still non-inferior. Should be defined a priori • Allows for smaller studies.

18. Power Analysis • Depends on: • Superiority,equivalence, and non- inferiority • Difference between the two groups that will cause the null-hypothesis to be discarded-i.e. what difference do you want to detect. • Usually estimate what is the inherent difference likely to be between the two groups. • What is the variability within each of the groups. • The standard deviation • Type of outcome measure • Continuous variable, dichotomous, survival time, etc.

19. Power Analysis Also depends on: • Allocation ratio • 1:1, 1:2 etc. • Defined acceptable power to detect a difference (usually 80%). • Defined acceptable rate of detecting a difference in error (usually 5%).

20. Study Design ConsiderationsReporting results • Major journals have adopted standards for reporting results of randomized trials, the CONSORT Statement* • Any of these are applicable to any therapeutic trial, even when not randomized. • Our literature could be substantially improved *Maher D, Schulz KF, Altman D. The CONSORT Statement: Revised recommendations for Improving the quality of reports of parallel-group randomized trials. JAMA 2001;285:1987-1991

21. Study Design ConsiderationsCONSORT Statement • Participants eligibility stated at the time of study design, along with exclusion criteria. • The interventions described in detail. • Primary and secondary outcome measures defined. • Statistical analysis defined prior to start of study • Maher D, Schulz KF, Altman D. The CONSORT Statement: Revised recommendations for • Improving the quality of reports of parallel-group randomized trials. JAMA 2001;285:1987-1991

22. Reporting ResultsCONSORT Statement • Flow of patients through study provided in flow chart. • Number screened, qualified, randomized, treated, and assessed at each stage of follow-up. • Dates of recruitment provided • Baseline demographic and clinical charateristics provided.

23. Study Design ConsiderationsCONSORT Statement • Intention to treat analysis • Based on how randomized, regardless of whether actually treated as randomized. • A key component of internal validity: It is inherently unbiased. • Provides an insight into the desirability of the two treatments- non-adherence is not random. • If use “as treated” analysis, will be excluding patients who may be different from those who proceed. • Their analysis will be distorted

24. Study Design ConsiderationsCONSORT Statement • Adverse events reported in both groups. • Ancillary analyses take into account multiplicity of analyses. • Less weight put on conclusions from secondary analyses. • Discussion includes: • Interpretation of results and generalizability • Discussion of limitations, potential biases. • Discussion of the evidence in context of other studies, current knowledge.

25. Summary • Key aspects of internal validity in a therapeutic trial: • Population described, flow of subjects recorded. • Power analysis to determine proper number of subjects. • Random allocation. • Blinded assessment of outcome. • Intention to treat analysis.

26. Summary • Randomized trials need to become the standard by which we evaluate new therapies. • Usually should be assessed compared to current standard therapy. • CONSORT Statement guides reporting for most journals, but is a useful tool to for designing and reporting even non-randomized studies.