Statistical Concepts of Clinical Trials

1. Statistical Concepts of Clinical Trials M. Hasan Rajab, M.S., Ph.D., M.P.H. Alfaisal University & Texas A&M Health Sciences Center Clinical Research Training Workshop

2. Session Objectives • To introduce participants to basic concepts useful in understanding of study design issues. • To introduce participants to important aspects of data analysis and results interpretation. • To help participants appreciate the applications of biostatistics to their work. • To introduce participants to ethical issues essential to their research projects.

3. Topics • Introduction/Motivation • The research process: An Overview • Study Design • Hypothesis • Selection of end points and response rates • Determination of sample size • Randomization • Data Analysis Issues • p-value vs. CI • Clinical significance vs. statistical significance • Ethical issues, data and safety monitoring (DSMB)

4. Introduction & Motivation • Clinical trials are the foundation for the development of new drugs and therapies for the detection, monitoring, prevention and treatment of cancer. • A carefully designed and efficiently conducted clinical trial can produce results that change clinical practice, and deliver new drugs. • A poorly executed trial consumes precious clinical and financial resources and challenges the validity of the ethical agreement between investigators and participants.

5. Misapplication of Statistics • "The statistics on sanity are that one out of every four Americans is suffering from some form of mental illness. Think of your three best friends. If they are okay, then it's you!"-- Rita Mae Brown

6. Introduction • We plan to focus mainly on clinical trials. • However, it is widely accepted that the basis for patient care decisions should not be restricted only to RCT or meta-analysis study designs. • There are uncommon diseases or complex pathologies that can’t be investigated using above study designs, i.e., ok to use designs that achieve weaker levels of evidence, such as cohort studies, outcomes research, and case-control study. • Reference: Virtualmentor.ama-assn.org/2011/01/pfor2-1101.html

8. The Research Process

9. Study Design • Appropriate statistical methods for study design, and data analyses of research studies are essential in order to obtain unbiased results • The main objective of study design is …to reduce bias. • So, what is bias?

10. Bias • It is the deviation between the observed (sample average) and the actual (population average) values • Is it bias or trend? • They are different. However, bias is more critical • Learn to distinguish between them. • Types of bias: • Selection • Recall • Other...

11. Precision • Larger samples give more precise (accurate) estimates • Study cost usually increases with sample size • However, cost increases at a much faster rate than precision. • Confidence intervals (CIs) quantify the precision of the statistical estimates (e.g., mean) we have computed from a sample. • What is the relationship of bias and precision?

13. Study Design – PARC Method • How many of you use the PARC method (Plan After Research Completed) for designing an experiment? • That is, first conduct the experiment, and collect the data, then try to determine what questions can published. • The usefulness of this method is best described by spelling its name backwards.

14. Study Hypothesis • Research studies are initiated with research questions which are linked to the development of formal study objective(s). • Next, investigators formulate hypotheses to best achieve study objectives. • Definition: Hypothesis is an assumption written in a form that allows it to be tested or challenged.

15. Research Hypothesis • To assess the validity of a hypothesis, it must be testable. • Example 1: “The study hypothesis is that high dose omega- 3, given orally to arthritic patient, will reduce joint inflammation”. • Example 2: “The study hypothesis is that high dose omega- 3, given orally to arthritic patient, might reduce joint inflammation”. • Which hypothesis is testable? Note: “will” – can be wrong or right, however, “might” you can be right either…”, i.e., not testable.

16. Selection of Endpoints • Definition: An endpoint is a measurable outcome that signifies an intervention's effectiveness. • You must have a well defined endpoint, i.e., to study an idea would be an important goal; how will you know when you have “studied” this idea enough?!

17. Examples of Endpoints • Tumor response rate • The proportion of participants whose tumor was reduced in size by a specific amount; e.g., if 7 of 10 patients responded, the response rate is 70%. • Disease-free survival • The amount of time a participant survives without cancer occurring or recurring, measured in months. • Overall survival • The amount of time a participant lives, measured from the beginning of the clinical trial until the time of death.

18. Selection of Endpoints • Selection of endpoints depends on the phase of trial. For instance: • Phase II: Tumor response rate is usually selected. • Since tumor response rates are often temporary and may not translate into long-term survival benefits, response rate is a reasonable measure of a treatment's effectiveness in a phase II trial. • Phase III: Participant survival and quality of life are usually selected.

19. Selection of Endpoints • Selection of endpoints also depends on the type of trial. For instance: • Treatment trial endpoints • Participant survival • Tumor response rate • Quality-of-life trial endpoints • Participants' welfare • Control of symptoms

20. Sample Size: Justification • The number of participants in a research study greatly influences its statistical significance. • With too few participants, a research study does not generate enough information to draw an appropriate conclusion. Hence, important findings may be missed. • But, by testing more people than needed to obtain statistically significant results, a study may.. • take longer to produce results, or • give ineffective or unsafe therapy more than necessary.

21. Why Estimate Sample Size Early On? • To know if there are enough eligible patients available that could be enrolled in a reasonable time period. • To know whether there are enough resources ($$$) available to complete the study. • Other…

22. How Large Should the RCT Be? • The study needs to be large enough to have a high likelihood of detecting a clinically important difference between studied treatments; otherwise, the patients that are enrolled may have been done so in vain.

23. Sample Size Estimation • How many patients do I need for my clinical trial? • When planning a clinical trial, researchers first decide how large a difference between treatment groups is clinically important. • Based on this and other information they calculate sample size, or how many people should be enrolled in the trial.

24. An Example: Comparing 2 Proportions p1 = Projected proportion of successes in Group 1 p2 = Projected proportion of successes in Group 2 Null Hypothesis, Ho: p1 = p2

25. Sample size formula to compare two proportions • Determination of sample size Note: This gives the sample size per group

26. Sample Size: Comparing 2 Proportions • The chart below gives the number of subjects (per group) required to detect a difference between P1 (left column) and P2 (top row) at 5% level of significance (LOS). • With 90% Power: P2 • P1 .10 .20 .30 .40 .50 • .05 582 101 47 28 19 • .10 266 82 42 26 • .20 392 109 52 • .30 477 124 • .40 519

27. Sample Size: Comparing 2 Proportions • With 80% Power (at .05 LOS): • P2 • .10 .20 .30 .40 .50 • P1 • .05 435 76 36 22 15 • .10 199 62 32 20 • .20 294 82 45 • .30 356 93 • .40 388

28. An Example: •  Assume a primary outcome measure of a study can be expressed using a binomial response (Yes/No, Alive/Dead, Success/ Failure, etc.). • To obtain an initial estimate of the sample size, the clinician needs to provide information that may come from experience, the literature, or simply an expert opinion, (an educated guess).

29. An Example, …continued • An oncologist is investigating a new treatment for prostate cancer in a patient population. A new drug has been reported in the literature, and the oncologist wants to test it against the current standard medication. • After consulting with colleagues, the oncologist determined that available standard treatment is effective in 30% (P1 = 0.30) of patients . • The oncologist also decides that, all other things being equal, he (she) would switch to the new medication if the treatment response rate improved from 30% to 50% (P2=0.50). • These figures werereported to the study biostatistician who provides a sample size estimate of 93 patients per group, assuming 80% power, and 5% LOS.

30. Sample Size: Comparing 2 Proportions • With 80% Power (at .05 LOS): • .10 .20 .30 .40 .50 • .05 435 76 36 22 15 • .10 199 62 32 20 • .20 294 82 45 • .30 356 93 • .40 388

31. Sample Size Computations Comparing 2 Means m1 = expected mean in Group 1 m2 = expected mean in Group 2 d = m1 – m2 s = expected standard deviation

32. Sample Size Formula For a = 0.05 and b = 0.10 (90% power) we have:

34. “Head in the Sand” Approach • Forget about sample size, do the study and see how many patients turn up. • This approach may result in a small trial with little scientific merit.

35. Randomization

36. Randomization • In statistical theory randomization means randomly allocating the experimental units across “cancer” treatment groups.

37. Random vs. Haphazard • Random sampling: Every member of the population has the same probability (equally likely) of being selected into the sample. • Non-random sampling: Not every member of the population has the same probability of being selected to the sample (e.g., convenience sampling where individuals are selected because they are in the right place at the right time).

38. Randomization & Bias • The two major techniques to reduce bias in RCTs are randomization and masking (blinding). • Randomization provides safeguard against selection bias, and in the long run against accidental bias.

39. Randomization • Randomization creates treatment groups that are similar at the baseline, i.e., starting point, on both known and unknown factors associated with the type of cancer being studied. • It quarantines that a participant has an equal chance of being assigned to one of two or more groups.

40. Randomization: How to? • In phase III trials (and some phase II) participants are assigned to either the investigational or control group via a computer program, or table of random numbers. • For example, if an oncologist wants to compare a new drug to treat prostate cancer against a standard drug, then study participants should be randomly allocated to either the new drug (treatment) or to the standard drug (control).

41. Statistical Significance & p-values • Data is usually analyzed to determine the statistical significance, which is the extent to which findings differ from the chance occurrence. • We usually use two levels of significance; 0.05 or 0.01. • Statisticians use a term called p-value to quantify the level of the statistical significance.

42. p-value • p-values represents the probability the results of a clinical trial are due to chance rather than due to a real difference between the tested treatments. • The smaller the p-value, the greater the likelihood that the results are not due to chance. • For example, a p-value of 0.05 (i.e., 1 in 20) or smaller is widely accepted as an indication that the results are statistically significant, i.e., the study results have a probability level less than 0.05 of occurring by chance.

43. Clinically Important Difference • It is the smallest treatment difference that is of clinical value. • We usually define what is clinically important difference at the planning stage.

44. Statistical vs. Clinical Significance • Statistical significance does not imply clinical or significance. • A study can have: • Both statistical and clinical significance • Statistical significance only • Clinical significance only • Neither

45. An Illustration • The result of an oncology clinical trial can be statistically significant (difference was not due to chance) without being clinically significant (medically important). • Assume a group receiving an experimental treatment has 2% higher survival rate than a group receiving the standard treatment. This difference could be statistically significant, but if participants who survive longer experience serious side effects, it may or may not be medically important. • In this case, the side effects might be worth tolerating only if the experimental treatment group has a 10 % survival rate. • Good trial planning and interpretation take into consideration both medical importance and statistical significance.

46. Ethical Concerns • There is a need to establish a system for the appropriate oversight and monitoring of the conduct of cancer clinical trials to ensure the validity and integrity of the data & safety of participants. Components of this system include: • http://www.nidcd.nih.gov/research/clinicaltrials/datasafety.asp#policy • Report adverse event to the IRB on time • Conduct interim analysis,… but how often? • Declare any conflict of interest!

47. Data and Safety Monitoring Board (DSMB) • It is an independent committee whose membership includes, at a minimum, a statistician and a clinical expert in the area being studied. • Other members are experts in all scientific disciplines needed to interpret the data and ensure participants’ safety.

48. DSMB • The objectives of data and safety monitoring plans are to: • Ensure that risks associated with participation are minimized to practical and possible extents. • Ensure data integrity. • Stop a trial if safety concerns arise or if its objectives are met earlier than expected. • The NIH requires all phase III clinical trials to undergo monitoring by a DSMB, and that all phase I and II clinical trials have a data and safety monitoring plan.

49. Ending Trials Early • There can be compelling reasons for halting a trial early. • For example, if participants experienced severe side effects, or if there was a clear evidence that risks outweighed benefits, the DSMB will recommend that the trial be stopped early. • A trial might also be stopped early if there is a clear evidence that the new intervention is effective, in order to make it widely available.

50. An Example • The Breast Cancer Prevention Trial, conducted by NCI's National Surgical Adjuvant Breast and Bowel Project, was designed to evaluate whether taking the drug tamoxifen could prevent breast cancer in women considered to be at high risk of developing the disease. • In March 1998, interim data showed that tamoxifen cut the chance of getting breast cancer almost in half. • Instead of continuing the trial for the full 5 years, as planned, researchers stopped the trial after about 4 years. • Women in the trial who were taking tamoxifen were offered the opportunity to continue treatment for the remaining 14 months of the trial. • Women receiving the placebo were invited to participate in the Study of Tamoxifen and Raloxifene, or STAR trial, designed to determine whether the osteoporosis prevention drug raloxifene is as effective as tamoxifen in reducing the chance of developing breast cancer. • The women's other option was to seek tamoxifen from a physician on their own, outside a clinical trial.