Using an Independent Statistician to Support a Data Monitoring Committee

1. Using an Independent Statistician to Support a Data Monitoring Committee Patrick D. O’Meara, Ph.D. Pat O’Meara Associates, Inc. pat@patomeara.com FDA/Industry Workshop 14-16 September 2005

2. Outline • Introduction • Checklist • 2 examples • Recommendations

3. Introduction “. . . the integrity of the trial is best protected when the statistician preparing unblinded data for the DMC is external to the sponsor, especially for the critical studies intended to provide definitive evidence of effectiveness.” - Draft Guidance On the Establishment and Operation of Clinical Trial Data Monitoring Committees

4. Introduction There’s more to protecting the integrity of the study than just engaging an independent statistician to perform a survival analysis for the DSMB.

5. Checklist • Charter • Contract • Documentation from Sponsor • Database • Support from the Sponsor

6. DSMB Charter • Independent statistician’s role • Who directs the IS work? • Sufficient freedom to create tables/analyses requested by the DSMB • In effect the IS acts as an employee of the DSMB • Roles of other organizations who provide data • CRO • IVRS • Central clinical lab • Biomarker lab

7. Contract • Tasks assigned to the IS • Programming of the tables, graphs, & listings • Who will write them? • How will they be verified ON THE IS’s SYSTEM? • If written by the Sponsor, what changes are allowed? • Who is in responsible for the data? • Especially important if a CRO has been retained to monitor the study and to prepare the data for interim analysis: • Adding variables to an analysis database • Correcting errors, inconsistencies • Interpretation

8. Contract • Time to assemble the report • From time IS receives the data • Once agreed Sponsor must not let it slide. • Time to prepare a presentation • Especially important at formal interim analysis when auxiliary analyses may be needed to support result.

9. Documentation from Sponsor • Protocol & blank CRF • Informed consent forms • Formal Statistical Analysis Plan • Includes a clear statement of decision rules for any interim analysis: • Null and Alternative Hypotheses • Significance level • Test statistics and methods • Futility, efficacy • Adjudication process & rules • Investigator’s Brochure

10. Documentation from Sponsor • Proposed list of Tables, Graphs, Listings • Any special consideration in the studies • The definition and processing of SAEs in a clinical endpoint study • For example, stroke • Biomarker lab data

11. Database • Sources of the data • CRF database (SAS) • Analysis database (SAS) • Clinical laboratory (SAS, text) • IVRS (SAS, text, or Excel) • Biomarker or specialty laboratory (text, Excel) • SAE database (expedited review – pharmacovig.) • Current death list (Excel) • 24-hour reports of SAE/Clinical endpoints

12. Database • Detailed specifications – derivations of derived variable • Annotated CRF • Description of special processes • Topics that Sponsor and IS should discuss • Frequency of updates • Timing before reports • Robustness of interim cuts of the database • Do AEs disappear? • Are some data sources more reliable or current than others?

13. Support from the Sponsor • Availability by telephone or email of • A Statistician who can speak with authority about the study and proposed analysis. • A Statistician/Data analyst/Programmer who can answer detailed questions about the data. • Face-to-face meeting with key project team members data management and statistics • Learn the system that produces the data for DSMB • Especially useful when resolving inconsistencies.

14. Example 1 • Two treatments; Planned subjects: ~2200; • 28-day all-cause mortality • Two interim analyses planned • Protocol, SAP • Tables, listings from Sponsor’s standard library • The Job: IS to reproduce TLG and present • Randomization schedule from IVRS group • 4 data sets for analysis with detailed specification • 8 days to prepare and ship report

15. Example 1 • Statistician assigned to database quality • Sponsor’s project statistician provided SAS code to implement formal interim analysis. • A dummy r.s. using A: odd, B: even • 4-5 test shipments of data before 1st interim • Using last test shipment, 100% check of all tables against set produced by Sponsor.

16. Example 1 • 1st Interim Report and Analysis • Timeline for report was squeezed by 1-2 days • DSMB decision to continue study without change • 2nd Interim Report and Analysis • Efficacy was demonstrated. • Auxiliary analyses demonstrated consistent trends across many subgroups • DSMB recommended stopping for efficacy • IS presented results to Sponsor executive comm. • Study stopped.

17. Example 1 • Lessons learned • Well-defined roles and responsibilities contributed to team environment. • Extra data transfers allowed practice so that final transfers went smoothly and tight timelines could be met. • Time spent reviewing database was a big contributor to success of the project. • 100% check of tables using dummy r.s. was essential.

18. Example 2 • Two treatments; ~1800 patients • All-cause mortality • DSMB meetings every 3 months at beginning • Protocol, no SAP • Review imbalance every 8 deaths for 1st 100 • IVRS, Sponsor’s DM, Biomarker, Clinical lab • Monthly updates of clinical database via FTP • Sponsor provided SAS programs for data from Clinical database (primary motivation $$)

19. Example 2 • Programs designed for VAX; local system PC. • File references in every program had to be changed. • Each program contained an extensive block of code to merge in randomization schedule. • Logic errors in several programs. • After a month, client resent programs with changes but the changes were not documented and all the file references had to be changed again.

20. Example 2 • Meanwhile, Biomarker data had been coded by the lab to prevent inadvertent blinding. • Coded results were manually transcribed into database – no source record of original value. • Coded with C++ algorithm using radix-32. • When decoded found that there had been many transcription errors --- invalid values.

21. Example 2 • DSMB expressed concern about decision rule for stopping for safety. • Requested Monte Carlo study. • Decided to meet again in 10 days to discuss. • Monte Carlo showed that under reasonable assumptions, chances of stopping were less than .05. • Recommended changing rule so Pr(Stopping) ~ .15. • Next DSMB, no improvement in safety – study stopped.

22. Example 2 • Lessons learned: • Transferring SAS programs between systems requires careful planning and lots of work. • Need to specify who is responsible for correct implementation on local system. • Nothing in contract that said IS could do additional analyses suggested by DSMB. • Serious decisions by DSMB could result in liability. • Unforeseen problems can heavily influence the amount of time spent on a project.

23. Recommendations • Provide the independent statistician with all the available information about the study: • Protocol • Statistical Analysis Plan • Assign project team members to with the IS. • Charter should state IS works for the DSMB. • IF Sponsor decides to provide programs, • Work with IS when designing • State in contract & charter who is responsible • Contract should indemnify IS.

24. The End