Protocol Development and Statistical Analysis Plans

1. Protocol Development and Statistical Analysis Plans Petra Rauchhaus TCTU Clinical Trials Statistician

3. Importance of the Protocol Funders Trialists Ethics Journals Patients • Provide rationale for the trial • Define trial goals and processes • Define methods of analysis/ reporting • Enable scientific and ethical review • Provide a “Trial Roadmap” Systematic Reviewers Healthcare providers Policy makers Ethics Journals

4. Importance of the Protocol • GCP Requirement • Ethics Committee requires a protocol for submission • Part of the EU Clinical Trials Register (EUDRACT) • Ensures in Multi-Centre Trials that all centres perform the study in the same way • Journals require a registered protocol for publication • Not only for CTIMPS, Non-CTIMPS also benefit from a good protocol

5. What could go wrong? Missing details of basic trial design (uncontrolled/ controlled/ randomized) Imprecise or missing description of the primary outcome in the protocol Sample Size calculation not reported Limited methodological information Interventions not well defined Planned subgroup analyses missing Favourable reporting of positive outcomes Adverse events suppressed in reports

6. Lack of general information Allocation Concealment Blinding Primary Outcome Power Calc. Adverse Events Reporting System Chan AW et al, BMJ 2008; Al-Marzouki S et al, Lancet 2008

7. Lack of statistical information Handling deviations Primary Outcome Analysis Adjusted Analyses Subgroup Analyses Handling Missing Data Chan AW et al, BMJ 2008; Al-Marzouki S et al, Lancet 2008

8. Protocol standards There is a number of support documents: • ICH Guideline E6 defines the protocol structure (15 sections with several sub-points each) • SPIRIT (Standard Protocol Items for Randomized Trials) initiative by statisticians, journal editors and PIs • CONSORT guidelines to report trials • EQUATOR Network:http://www.equator-network.org/ • TASC SOP 14: Writing a protocol • Protocol Template on the TASC website:http://www.tasc-research.org.uk/_page.php?id=208

9. Definition of a protocol Pre-Trial Document containing transparent description of: • Background and objectives • Population and interventions • Methods and statistical analysis • Ethical and administrative aspects

10. Title • A title uniquely identifies the project • It should summarize the aim and methods of the trial • Important information (e.g. randomized, double-blind, parallel group) should be included in the title • Indexers on websites such as PubMed may not classify a report correctly if the authors do not explicitly report information in the title • “A Prospective Randomized Study of Medial Patellofemoral Ligament (MPFL) Reconstruction “

11. Synopsis • Brief overview over the study aims and conduct • Should contain sufficient information about a trial to serve as an accurate record of its conduct • Should accurately reflect what is included in the full protocol and should not include information that does not appear in the body

12. Background • The Declaration of Helsinki states that biomedical research involving people should be based on a thorough knowledge of the scientific literature • Thus, the need for a new trial should be justified in the introduction • Explain the scientific background and rationale for the trial • Report any evidence of the benefits and harms • Ideally, it should include a reference to a systematic review of previous similar trials or a note of the absence of such trials

13. Objectives • Objectives are statements what the researcher means to do • Objectives can be seen as smaller problems in the larger research area • E.g. “Improving cancer care” is a large research area which is too broad to be tested within a trial.Impact of physiotherapy on QOL of late stage lung cancer patients is testable within a trial. • Ensure that objectives are specific, measurable and clinically important • Changing objectives can sometimes make a trial better

14. Outcomes • Is the measurable part of the objective • Ensure that the outcome is appropriate to the objective it serves. • Define clearly what the outcome is and how it will be measured • If outcomes are measured several times, specify time point of interest • If possible, use validated and measurable outcomes • If there is more than one assessor, state how many there are and how discrepancies in measurement will be handled

15. Trial Design • Define the type of trial, e.g. parallel group, cross-over or factorial • Define the conceptual framework, e.g. superiority, non-inferiority, equivalence or other • If a less common design is employed, authors are encouraged to explain their choice • This is especially important because it might have implications on sample size or analysis • Include allocation ratio if more than one group, and unit of allocation (patient, practice, lesion)

16. Eligibility Criteria • Should be well defined and appropriate to the trial • Define which patient groups are involved and how they relate to the objectives • Eligibility criteria which are too narrow can jeopardize the study • Eligibility criteria too wide can invalidate the outcomes • E.g.: Including Stage IV Cancer patients in a study examining the effectiveness of two different treatments might fail, as the diseases is too advanced already to make a difference

17. Sites and Locations • Goes hand in hand with the eligibility criteria, as certain subjects need certain locations • E.g.: primary care, hospital wards, specialized units • Healthcare institutions vary in their organisation, experience, and resources • Social, economic, and cultural environment and the climate may also affect a study’s validity • Especially important in multicentre trials, particularly in international studies

18. Interventions • Describe all interventions including controls in great detail • It must be possible to be reproduced if necessary • If you compare to “usual practice” describe what that means, do not assume everyone knows • If interventions are variable, e.g. adaptation of radiation doses or drug regimes, define rules of application • In dose-escalation studies, define stopping rules

19. Sample Size • Sample size calculations are based on previous trials measuring the outcome • Ensure that the patient population matches the trial population • Where no previous trials are available, sample size is often based on assumptions • Sample size is only as accurate as the assumptions • Where more than one outcome is present, sample size is calculated for the primary outcome usually • It is possible to use the largest sample size to get the best power

20. Interim Analysis • Interim Analysis can diminish the trial power • Error rates increase as the number of analysis increases • E.g. doing 5 interim analysis requires a p-value of 0.01 rather than 0.05, and can give an error rate of 19% rather than 5% • Use only when necessary • Some trials require interim analysis, e.g. for a DMC • If possible, separate the DMC analysis from the main analysis

21. Randomisation • Randomized trials are the gold standard • Randomization requires a program to be written • Sequence generation must be reproducible at any stage • Define criteria for stratification and minimisation • Try to avoid predictable block sizes • If possible, blinding should be employed • Blinded studies require an independent statistician • Minimization is dynamic and therefore less predictable

22. Allocation • Allocation concealment is not blinding • Define how the allocation is applied to the subjects • Define how to conceal allocation until the subject is included into the trial • Ensure that the person doing the screening is not familiar with the allocation sequence • Decide whether to include a subject into the trial before the allocation • If possible, use a third party to allocate subjects

23. Statistical Analysis • Statistical analysis must be described • Descriptive statistics should be defined for an overview over the data • Define the appropriate methods for the data • Describe briefly missing or spurious data • Keep the description of the statistical analysis short • Mention checks of normality and independence • Do not hesitate to involve a statistician with this part of the protocol • A detailed statistical analysis plan (SAP) should be written during the course of the trial

24. Statistical Analysis Plan (SAP)

25. Statistical Analysis Plan (SAP) • It is critical link between the conduct of the clinical trial and the clinical study report.  • General statistical analysis is defined in the clinical protocol • The SAP contains a more technical and detailed elaboration of the analysis • Recommended by the CONSORT guidelines and ICH Guideline E9 (Statistical Principles for Clinical Trials)

26. Why write an SAP? Establish Good StatisticalPractice Implement the Trial as outlined in the Protocol Study Design (Clinical Protocol) Study Methods (Data Collection Trial conduct) Study Analysis provides checks on the original design Study Analysis(SAP) Analysis of the planned study design, adapted tothe study methods

27. GCP requirements • The statistical authorship of the SAP should be clear • Version and date should be clearly defined • The SAP should be reviewed/ updated immediately before the blinded code is broken or before analysis begins in an unblinded trial • The SAP should be signed off by the PI/ CI and the Statistician (and other members of the study team where applicable) • Changes in the SAP after study end should be justified and fully documented in the statistical report

28. When to write an SAP • The SAP is written during the trial, after the clinical protocol is final • It must be finalized and signed off before the end of the trial to avoid bias • If the study is blinded, it must be finalized before the blind is broken • The SAP should be reviewed and possibly updated as a result of the blind review of the data • In adaptive trials, it must be finalized before the first interim analysis • Regulatory factors, such as a special protocol assessment at the FDA, may affect the timing

29. Changes in Study Methods • Protocol Amendments during the trial • Change in the planned treatment (new developments in therapy or guidelines) • Recruitment does not go as planned • Early termination of the trial can change patient numbers • Adding or removing a group • Addition or removal of a planned test or procedure • Changes in the outcomes or how they are measured

30. SAP Contents • A brief description of the purpose • The study rationale as laid out in the protocol • Definition of analysis populations (usually ITT) • How subject data will be summarized (descriptive statistics or counts/ percents) • Which statistical tests will be used on which data • The statistical methods to be used for the endpoints • When and how to impute missing or partial data • Mocks (or shells) of all unique TLF's • Quality control of the analysis

31. Writing an SAP • Refer to TASC SOP 05 (Statistical Analysis Plans for Clinical Trials of Investigational Medicinal Products)http://www.tasc-research.org.uk/_page.php?id=266 • Follow the section headings laid out in the SOP • Contact the study statistician if present • If no study statistician is present, TASC statisticians can review the SAP • Distribute the SAP to all members of the study team that can contribute • Finalize the SAP before the study is finished

32. Benefits • Clear Protocol and SAP show that a study was done according to GCP standards • Avoid biased analysis by defining the study populations before study end • Defined handling of missing data, outliers and data deviations make the analysis more transparent • Clearly defined subgroup analysis ward off data dredging • The study report and resulting papers will be more likely to be of high quality

33. Any Questions?