Some Clinical Trial Design Questions and Answers

Some Clinical Trial Design Questions and Answers Peter A. Lachenbruch Oregon State University

Usual Disclaimer • The views expressed here are mine. While this presentation was initially developed while I was at the FDA, the opinions may not reflect those of the FDA • There are lots of acronyms – FDA never met an acronym it didn’t like. • Please feel free to stop me and ask for clarification

Orientation • I discuss some questions regarding clinical trial design principles. There are many books on the basics of clinical trial design and analysis • Pocock • Freedman Furberg and DeMets • Piantadosi • Chow and Liu

Orientation (2) • FDA has many guidance documents on their web site www.fda.gov Consult these for further details. • The International Conference on Harmonization (ICH) has issued many reports that worldwide regulatory agencies will abide by. Web site: www.ich.org • See E9 (Statistical Principles for Clinical Trials), E10 (Choice of Control Group and Related Issues in Clinical Trials , E3 (Structure and Content of Clinical Study Reports), E6 (Good Clinical Practice), and E5 (Ethnic Factors in the Acceptability of Foreign Clinical Data) for particularly useful documents

Orientation (3) • Off-label use • Use of a product for a condition that it has not been approved for (i.e., not on label) • FDA does not regulate medical practice and there may be drawbacks – e.g., drug interactions, adverse events • Limited information from these uses • Investigational use • Product being studied (usually at a limited number of sites) for use for some indication. Many controls on studies. • Compassionate use – when products are close to approval, FDA may allow use for patients not in a clinical trial

Orientation (4) • FDA organization • Center for Biologics Evaluation and Research (CBER) – things like vaccines, blood, genomics • Center for Devices and Radiological Health (CDRH) – stents, wheelchairs, band-aids, TV radiation – mechanical stuff. • Center for Drug Evaluation and Research (CDER) • Others: Center for Veterinary Medicine (CVM), Center for Food Safety & Applied Nutrition (CFSAN), National Center for Toxicological Research (NCTR)

General Statistical Ideas • Clarity of approach • Full disclosure of design, sample size calculations • Analysis methods • Distinction between CONFIRMATORY and Exploratory analyses • Confirmatory analyses are specified in the protocol (and will lead to licensure) • Exploratory analyses are those that are suggested by the data (think of shooting an arrow into a barn and then painting the target around the arrow!) • If “new” methods are used, there should be peer-reviewed citation

Statistical Ideas (2) • Must be analytically appropriate • Maintain size (α level) • Maintain blinding as appropriate • Have endpoints (outcomes) that are appropriate • Show a clinical benefit • Reliable and valid • Minimize missing data and provide plan for dealing with them when they occur

Question 1 • What pitfalls do the FDA see when information from pre-clinical data (or early clinical data on a similar investigational product) is formulated into a Phase I protocol?

Q1: (1) • Safety issues • A main use of pre-clinical data is to ascertain basic safety information. If animal data is limited, the FDA may ask for further study • Carcinogenicity studies – are there excess cancers • Immunogenicity studies • Teratogenicity studies – do fetuses develop normally • The choice of animal model is important – if it is not accepted as appropriate, there may be need to obtain further information or information on another model (the lists below are not exhaustive • Rodents: mice, rats, rabbits • Non-primate vertebrates: dogs, cats, pigs • Primates: rhesus monkeys, macaques

Q1: (2) • Need to characterize the product • Potency assays • Purity • Identity • All specifications need to have at least a start at understanding leading to full GMP compliance (GMP=good manufacturing practice)

Q1: (3) • It is recognized that proof of concept studies in pre-clinical studies may be limited. However, there should not be evidence of poorer outcomes than comparators. • This means that the results are not significantly poorer, not that the mean in one group is lower than the mean in the other.

Q2: • What are the various types of study designs available when there are more than two comparators?

Q2: • Depends on purpose of study • Testing 2 or more dosage levels versus control • Parallel group design – may wish to account for ordering of dosages in the analysis • Testing both schedule and dose • Factorial design (high and low levels of each dose crossed) – allows examination of interactions in the analysis

Q2: (2) • Test amount of adjuvant and dose • Factorial design • Crossover designs are cannot be used in vaccine trials because the immune system is permanently affected (or at least affected for a long time). Thus, carry over effects are always present • Smallpox immunization may last for 20 years or more, and may be refreshed by exposure to the virus • Influenza may have rapidly waning immunity, and different strains may appear each year. There may be some cross-protection.

Q3: • What controls are appropriate when there cannot be any blinding in the trial?

Q3: • Almost any control is reasonable: placebo, standard of care • Compare treatment with control • In some cases, historical controls may be used, but these are rare • A historical control is a group that was observed previously. Problems with concurrency, lack of comparability, etc. • Generally not recommended

Q3 (2) • The endpoint / outcome variable that is being used and how it’s evaluated is most important • A subjective endpoint is usually a problem, so FDA expects that a blinded evaluator will be used in these cases. • An objective endpoint (e.g., confirmed disease by laboratory measures) is preferable • Survival is always objective, but may have too few events.

Q4: • What are the problems seen by the FDA with randomization in trials?

Q4: • Randomization is absolutely essential in vaccine trials • Issues • Cheating: unblinding the treatment assignment – need to have robust way of preventing this • Stratification: too many strata make it unlikely that there will be sufficient numbers in each stratum for precise estimation. The number of strata is the product of the number of levels in each stratum (Sex (2), Age (4), Ethnicity (3) = 24 strata)

Q4: (2) • Issues (continued) • Inadequate number of strata • age<2, 2 ≤ age <12, 12 ≤ age < 18, 18 ≤ age < 50, 50 ≤ age often important in vaccine studies • Introduction of bias • Not accounting for the design of the study in the analysis – just because you have stratified, you still must account for the stratification in the analysis.

Q5: • What are the steps in designing a dose-ranging/dose-escalation study?

Q5: • Goals: • To establish maximum tolerated dose (MTD), dose limiting toxicities, and/or maximum feasible dose • To establish minimum effective dose • Designs • One dose per subject, gradual increase by fixed amount (typically half log increases, with rules for stopping) • What’s the right starting dose?

Q5: (2) Dose ranging / dose escalation • Multiple doses per subject for short (3-7 days) or long (1-4 weeks or greater) periods • What is range that generates useful levels of antibodies? What level has adverse events? • Dose Escalation • Give successively larger doses or number of doses (booster doses) until subject responds • May not be helpful with vaccines because of permanent effect of a vaccine, but can use different subjects. In this case, subjects should be randomized to dose.

Q5: (3) • For a vaccine both dose and schedule need to be determined • A factorial design may be useful • Test all doses and all schedules • Can look for interactions to see if the response is additive or not • The specific adjuvant may be important • This may be expanded to look at a response surface • Useful for first trials to pick a dose-schedule combination for later trials

Q5: (4) • In vaccine studies, it is important to establish the duration of protection. • In clinical trials, sponsor can follow subjects for a year or more and observe if there are any changes in disease incidence over that time. • Alternatively, the sponsor can obtain serum samples to determine the level of antibodies. • It is not always clear how the serum antibody level relates to disease incidence (called a correlate of protection).

Q6: • How does one deal with multiple variables that will affect the outcome measure (with an understanding of fixed randomization schemes and adaptive/dynamic randomization schemes)

Q6: • If there are strata, including study sites, these always should be included in the analysis model. • Common covariates include (if they are not strata) age, sex, ethnicity, disease stage. • Usual method is to conduct an analysis of covariance – some covariates may not be ordered, often they are.

Q6: (2) • The analysis of covariance does an analysis of variance on the adjusted response. • Assumptions: • Normal distribution of residual error • Can handle with a permutation test • Covariates are not affected by treatment – measure at baseline! • Parallelism – no interaction of treatment and slope – i.e. the response treatment rate of change is same for all covariate combinations

Q6a: Fixed and Adaptive Randomization Schemes • Fixed scheme • Same proportion assigned to each treatment group • Different proportion assigned to groups such as 2:1 or 3:1 • Smallest variance of treatment effect associated with 1:1 allocation, but it may be important to gain understanding of safety profile, so a more extreme allocation may be used. More extreme than 3:1 is not very useful and leads to much larger sample size

Q6a: (2) • Here are total sample sizes for α=0.05, =0.1, mean difference=1, =3 • 1:1 allocation 380 • 2:1 allocation 429 13% increase • 3:1 allocation 508 34% increase • 5:1 allocation 684 80% increase • Thus, the unbalanced allocation leads to a substantial increase in sample size and consequent budget

Q6a: (3) • Adaptive randomization • Next randomization depends on outcome of prior subjects in the trial • Need fairly early response in trial. May be possible with skin or other reactions (if they occur within a few hours of treatment), a bit less so with immunogenicity (outcome after first series of treatment that may take 6 months), unlikely with clinical outcome (occurs after series of treatments and a relatively long follow up period)

Q6a: (4) • Another form of adaptive randomization attempts to balance covariates (e.g., minimization) • This can be done with vaccine studies • Needs to have a measure of imbalance • Must adjust for covariates used in imbalance score • Potential for manipulation? • There is considerable disagreement among statisticians about the appropriate analysis model.

Q7: • What factors are used to estimate sample size?

Q7: What factors are used to estimate sample size? • Most popular question to statisticians • Tell me what n I need? It depends on the context of the study • Factors (using a two group test as an example) • Significance level (α, often 0.05) • Type 2 error (, often 0.2 or 0.1; 1- is the power) • Standard deviation of observation () • Difference in means (1 - 2 ) • Allocation ratio (1:1, 2:1, etc.)

Q7: (2) • Significance level, type 2 error and allocation ratio are relatively easy to determine • Mean difference and standard deviation are usually based on preliminary studies and may be quite uncertain. • It is useful to take these preliminary differences and halve them • What is really needed is the ratio of treatment difference to the standard deviation

Q7: (3) • In vaccines we often want to estimate the vaccine efficacy and find a confidence interval with a lower bound that gives us assurance the vaccine is working well Where IV is the incidence rate for the vaccine group and IC is the incidence rate for the control group Lower bound must be substantially greater than 0

Q7: (4) • It’s easy to find an expression for the confidence interval and then one sets the lower bound of the interval to the desired level (set in consultation with FDA) • Often ¾ of the observed VE, and VE is set by needs of clinical prevention. For example if the target VE is 0.8 (or 80%) the lower bound would be 60% • These are not absolute criteria!

Q8: • How does the investigator choose the margin of equivalence or non-inferiority (delta or ) in comparative clinical trials? • The object is to show that a new product is not poorer than the approved product by an clinically unimportant amount. • This is the concept for generic products – want to show a) works about as well; b) about as safe or more so; c) costs less

Q8: • Demonstrate that active control has assay sensitivity – that is, it consistently shows itself better than placebo – need evidence of this (the active control is the approved product). • Some conditions are so variable that a non-inferiority study isn’t feasible. These may include psychiatric conditions, some rheumatological conditions, etc. • Is the control an appropriate one? Compare to best licensed product, not worst if there are multiple options. • What is clinically important? Depends on context of disease

Q8: (2) • When citing a % difference for VE (or anything) be sure to clarify whether you mean 10 percentage points or 10% of the comparator • If comparator has a VE of 80%, do we want the estimated VE of the new vaccine to be 72% or 70% if we choose a 10% margin? Or do we want the relative VE (vaccine vs. control) to be at least 90% • It’s easy to become confused so it’s good to be specific

Q9: • How does the investigator deal with missing data?

Q9: • Don’t have any • Don’t have very much (under 5% is my initial break point) • Discuss ways of dealing with it prospectively!!! • Complete case analysis (ugh!) • Last Observation Carried Forward (LOCF) (also ugh!) • Mean values for replacement • Regression models for replacement • Imputation models

Q9: (2) • Types of missing values • Patient misses visit, and an ‘interior’ value is missing • Patient drops out, and a series of values at the end are missing • (Some) Covariates are missing in one or more visits

Q9: (3) • Last Observation Carried Forward – when patient drops out, observations from that time onward are replaced by the last observation. • Is almost always a problem for me • It ignores any trends in data • It reduces variability arbitrarily • Can change significance level substantially – in either direction.

Q9: (4) • Mean values imputation • Need to be sure you don’t increase the apparent sample size (i.e., replaced values don’t give a more precise estimate) • When observations are replaced by mean values, most statistics programs treat the replaced values like other values. This reduces the variability and ‘improves’ the power. • This doesn’t account for patient specific characteristics • This can be especially tricky if a long series of values is imputed • Mean of patient values or mean of other patients at that visit?

Q9: (5) • Regression models • Determine what variables are “good” predictors of the missing value – usually useful to have a small number of such variables so they won’t have a lot of missing value issues • Predict missing value using a regression model (try to show that variance of predicted value isn’t too big)

Q9: (6) • More sophisticated imputation models have been developed recently • Determine classes of similar patients (“propensity scores”) • Fill in missing values by selecting randomly from observations in the same class. • Do this multiple times, (5 to 10 is usually enough) • Analyze the data and pool results

Q10: • How does an investigator determine what data should and should not be included in analyses (especially in the cases of protocol violations, withdrawals and drop-outs)?

Q10: • The fundamental efficacy data set includes all subjects as randomized • Note this does not say “ignore patients who didn’t get medication” – doing this messes up the randomization plan and allows data shaping • This is called Intent to Treat (ITT) • Modified ITT relaxes this to patients who have had at least one treatment. It still messes up the randomization design.

Some Clinical Trial Design Questions and Answers