SDTM Implementation - Best Practices

1. SDTM Implementation - Best Practices Pantaleo Nacci, Head Statistical Safety & Epidemiology/PV Biometristi Italiani Associati (BIAS) Seminar: CDISC, SDTM and ADaM: Moving from theory to practice SAS Institute - Milano, 14 March 2014

2. Agenda The story so far Benefits of implementing SDTM Basic requirements Legacy data conversion: a missed opportunity? The way forward Conclusions

3. The CDR Project • In 2010 Novartis Vaccines (NVx) initiated a very ambitious project to pool together all legacy studies for which data are electronically available in-house, as well as all new EDC trials, within the framework of a Clinical Data Repository (CDR) • A slightly adapted variant of CDISC SDTM 3.1.2 was selected as common standard for final storage, analysis and reporting, which allows storing of study metadata as well • All legacy study data are being remapped to this standard in two phases, and their metadata retrieved (mostly from protocols) and entered, for a total of around 400 studies | BIAS Meeting | P. Nacci | 14 March 2014 | SDTM Implementation - Best Practices

4. The CDR Project (2) • All EDC panels have been redesigned to collect data in a format compliant with CDASH standards, which allows easy remapping to SDTM • After a careful and lengthy selection process among three candidates, the solution chosen as the basis of the CDR has been SAS Drug Development (SDD) as a hosted environment • The re-mapping is done using custom-written SAS programs rather than SAS Clinical Data Integration (CDI), as initially planned | BIAS Meeting | P. Nacci | 14 March 2014 | SDTM Implementation - Best Practices

5. The CDR Project (3) • (Re-)coding of adverse events, concomitant medication, medical history, etc. is performed in Oracle Thesaurus Management System (TMS) • Our home-grown Standard Reporting Software suite of validated SAS programs was adapted to run in the new SDD environment AND to use the new CDISC data structures • Last but not least, all our existing processes were migrated to the new platform, and several new ones mapped and documented | BIAS Meeting | P. Nacci | 14 March 2014 | SDTM Implementation - Best Practices

6. Two years later... • More than two years after the CDR went live, all new clinical studies are now running completely in it • Data get pushed automatically into the system at regular intervals • SDTM data are then obtained directly from CDASH dumps nightly, so that the latest version is always available online • Deviations from the NVx standard are dealt with using a set of study-specific ‘rules’, and approval by a technical committee is necessary • Standard Reporting Software has finally been tested on real data, necessary fixes have been applied and new enhancements are actively being worked on as we speak • Special techniques are being developed to speed up operations in SDD | BIAS Meeting | P. Nacci | 14 March 2014 | SDTM Implementation - Best Practices

7. Two years later... (2) • Phase 2 of the Legacy Data Conversion sub-project has started, and data from 191 legacy studies, dating back as far as 1992, are now available for use, of course in addition to those from CDR-native ones • Utilities to create and maintain complete and ongoing dynamic ‘oceans’ have been fully validated, and jobs to maintain them are scheduled to run nightly • Other utilities to create static ‘data poolings’ have been validated too, and are available for use • Several data poolings have now been created and released for analysis | BIAS Meeting | P. Nacci | 14 March 2014 | SDTM Implementation - Best Practices

8. CDR - The Full Picture Phasing out • TGROUPS • Lab norms • Randomization • Exclusions • Stat. Populations TMS Coding (AE, MEDHX, CMED) Submission Data sets Scanned CRFs (ICRF) Clintrial Paper CRFs Stat. Analysis Monitor listings Adapter eCRFs – EDC Periodic Updates CRO data, eDiaries Clinical Data Repository - SAS Drug Development CDISC Data sets Pooled analysis Ext. Labs CTMS Rapid Pooling Data Mining Signal Detection. Serology database Serology Lab Future wish: SAE ->EDC-> CDR-> PV DB SAE coding * PV Safety reports SAEs | BIAS Meeting | P. Nacci | 14 March 2014 | SDTM Implementation - Best Practices PV Safety database * Ideally all coding in 1 system

9. Agenda The story so far Benefits of implementing SDTM Basic requirements Legacy data conversion: a missed opportunity? The way forward Conclusions

10. Regulatory Requirements • It was in October 2007, while attending the PhUSE annual conference in Lisbon, that I heard for the first time about JANUS from an FDA speaker • The message was clear even then: the need to adopt non-proprietary data standards was not a question of ‘if’, but just of ‘when’ • It took longer than expected, but on 6th February 2014 FDA officially published several guidance documents for electronic regulatory submissions, with a 90-day window for public comment • Various CDISC data standards (CDASH, SDTM, SEND, ADaM) are specifically mentioned as required | BIAS Meeting | P. Nacci | 14 March 2014 | SDTM Implementation - Best Practices

11. Effects on Submission Reviews • Even after adopting CDISC standard more pitfalls might be present in (therapeutic) areas not well characterized • E.g., given the degree of latitude still left to the single companies by the gaps in the current standards as regards vaccines data, an FDA reviewer might be faced with widely differing structures (and possibly codelists) representing the same data but coming from different companies • E.g., in vaccine trials solicited AEs can be stored in AE, CE, FA, etc. • This might result in longer times for review, even with the help of JANUS or similar systems | BIAS Meeting | P. Nacci | 14 March 2014 | SDTM Implementation - Best Practices

12. Evolution of the NVx Data StandardsHow an unstable reference can ultimately be a liability • It is now almost exactly two years since the first native CDISC study data entered the CDR • Only once that milestone was reached it was finally possible to fully test the reporting software, which had been developed using mock data • As expected, many issues, small and not so small, surfaced • The next slide show an overview of the changes applied to the initial production version of the NVx SDTM standard until June 2012 • A few further changes, mostly adoption of new domains, have been applied since then | BIAS Meeting | P. Nacci | 14 March 2014 | SDTM Implementation - Best Practices

13. Evolution of the NVx Data Standards (2)Adapting to changing requirements is a must | BIAS Meeting | P. Nacci | 14 March 2014 | SDTM Implementation - Best Practices

14. Evolution of the NVx Data Standards (3)How a pilot phase can save a lot of time later • In practice, these are a few examples: • Information which in the old data structures was contained in one variable is now spread over several ones, e.g.: • EDTTEST = ‘HI_B_BRISBANE08_CC’ • LBCAT=‘IMMUNOLOGY’ • LBMETHOD=‘HEMAGGLUTINATION INHIBITION’ • LBTEST=‘INFLUENZA B BRISBANE/2008 CC AB ‘ • Re-assaying of samples, only identifiable using the date when an assay was run, was not accommodated for until a new TSTDTC variable was added in SUPPLB | BIAS Meeting | P. Nacci | 14 March 2014 | SDTM Implementation - Best Practices

15. Usage of SDTM Data • As mentioned earlier, two data ‘oceans’, inherently dynamic, are maintained within it and have been used multiple times to create static ‘data poolings’ • Complete ocean: contains all available data from studies which are not active anymore; data for newly completed studies are added to those of existing ones • Ongoing ocean: contains all available data for studies not yet completed, and is recreated from scratch every night • All AE/MH verbatims can be updated to the latest MedDRA version in one go • These ‘data poolings’ are used to support submissions, but also papers, etc., i.e., whenever there is the possible need to go back to a known status to perform additional analyses | BIAS Meeting | P. Nacci | 14 March 2014 | SDTM Implementation - Best Practices

16. Agenda The story so far Benefits of implementing SDTM Basic requirements Legacy data conversion: a missed opportunity? The way forward Conclusions

17. Basic Requirements - General • The scope of such projects must be well defined at the beginning, so that all necessary functions are involved from early on • Proper resourcing levels are needed from the beginning, and all interested systems must be identified and properly covered • Think boldly when looking at the big picture and drawing your long-term plans, but don’t be ashamed to take only the steps you can afford at any given time • It is extremely dangerous to have all knowledge on a certain topic concentrated in only one or two persons, and it can also create unnecessary bottlenecks at the wrong time • A target reference framework must be clearly identified early on • While it is possible to implement CDISC structures from scratch just looking at the documentation, choosing the right system will make it easier to implement, e.g., the inevitable tweaks linked to the peculiarities in your legacy data | BIAS Meeting | P. Nacci | 14 March 2014 | SDTM Implementation - Best Practices

18. Basic Requirements – Legacy Data • The criteria to include/exclude studies and the overall timelines should be well understood and shared by top management to avoid creating excessive or plainly wrong expectations • Putting study data together takes a lot of time, patience and attention to detail, even when most of the electronic data are already more or less in the right format • Finding actual evidence about old studies can be a challenge (!) • Original documentation (at least annotated CRF and protocol/CSR, including all amendments) must be available for all studies before the coding starts, to avoid misinterpreting the data • Metadata are important as much as data themselves, so, e.g., make absolutely sure your T-domains are correct • If at all possible plan for a pilot phase, it will most probably save a lot of time and grief later | BIAS Meeting | P. Nacci | 14 March 2014 | SDTM Implementation - Best Practices

19. Agenda The story so far Benefits of implementing SDTM Basic requirements Legacy data conversion: a missed opportunity? The way forward Conclusions

20. What About All Those Studies in the Archive? • We decided that data from all new studies would be collected and stored according to CDISC standards, but this still left us with data from over 400 studies lying in our archives gathering dust • In this case the skill set required for the remapping was different, needing both good knowledge of the new CDISC standards and a lot of experience with the old one(s) • As described earlier, the first step was to decide which studies to convert first | BIAS Meeting | P. Nacci | 14 March 2014 | SDTM Implementation - Best Practices

21. Interpreting Old Data • A data inspection step was then undertaken, with the aim to identify all the legacy datasets and variables used • In our case we could further split the studies in pre- and post-Clintrial adoption: • Pre-Clintrial ones were managed by several different CROs in the first half of the 90’s, so that today existing knowledge is extremely limited and fragmentary • Post-Clintrial ones are on the contrary based on relatively minor variations of the current internal standard, which is known and understood by most members of the programming group • Study data were either re-extracted again from Clintrial or retrieved from our electronic archives | BIAS Meeting | P. Nacci | 14 March 2014 | SDTM Implementation - Best Practices

22. Necessary Tools • We correctly identified the need to retrieve all protocols, protocol amendments, (annotated) CRFs and CSRs from the archives, but underestimated the difficulty of getting them, especially the very old ones • The initial programming approach, agreed with our Quality partners, was based on random sampling of results, but a radical change to double-programming was requested well into the process • This caused a sudden shortage of experienced programmers | BIAS Meeting | P. Nacci | 14 March 2014 | SDTM Implementation - Best Practices

23. Remapping Your Existing DataAnother source of possible issues for your SDTM implementation • The LDC Phase 1 exercise, restricted to 153 high priority studies, took considerably longer than planned, but why? • Not all studies had available protocol and CRF at the time programming started, forcing the team to a dangerous guessing game • The NVD data standard was still evolving rapidly, so that some important assumptions proved grossly wrong when new studies were designed and run in the new system, sometimes requiring extensive rewriting of almost validated programs • One domain in particular proved a nightmare, LB/SUPPLB, as several old test codes, as well as free text descriptions, were not included in the new so-called lab metadata tables | BIAS Meeting | P. Nacci | 14 March 2014 | SDTM Implementation - Best Practices

24. Remapping Your Existing Data (2)Still something you want to do? • Even with all these issues and delays, this is still one of the core business reasons why NVx started the CDR project • Safety signaling across vaccines, therapeutic areas and even single vaccine components can now be performed using data in the oceans • This is meaningful only for projects where all data are already available in SDTM | BIAS Meeting | P. Nacci | 14 March 2014 | SDTM Implementation - Best Practices

25. Agenda The story so far Benefits of implementing SDTM Basic requirements Legacy data conversion: a missed opportunity? The way forward Conclusions

26. Future DevelopmentsWhat’s next on the CDR plate • The current implementation of CDR is based on SDD 3.5, which does not exactly shine for speed • A new, revised version of SDD (4.3) has been out for a while, and we have been getting a feel for it for some time • The set of available CDISC standards currently implemented is restricted to CDASH 1.1 and SDTM 3.1.2 • Two main tasks have been identified here, upgrading to SDTM 3.1.3/3.2 and adopting ADaM 2.1 • The Standard Reporting Software we use is still a revamped version of very old programs ported to SDD+SDTM • We are in the process of redesigning the reporting system from scratch, embedding in it the creation of ADaM datasets | BIAS Meeting | P. Nacci | 14 March 2014 | SDTM Implementation - Best Practices

27. Future Developments (2)What’s next on the CDR plate • Until now we have remapped only part of the available legacy data • Lessons learned from the Phase 1 have been put to good use to re-engineer the process • Many things have changed to the better in the CDR environment: • The reference standard is much more stable • OpenCDISC consistency checks are readily available to check the progress all along • Basic documentation has been retrieved for all remaining studies (no more guessing games!) • Programmers know better both sets of data structures | BIAS Meeting | P. Nacci | 14 March 2014 | SDTM Implementation - Best Practices

28. What is Happening on the SDTM FrontLately CDISC has been very active too! • SDTM 3.1 was released on 14 July 2004, SDTM 3.1.1 on 26 August 2005 and SDTM 3.1.2 on 12 November 2008 • After a hiatus of almost four years, during which more and more companies have adopted it, SDTM 3.1.3 has been finally released on 16 July 2012, followed by an utterly unexpected version 3.2 on 26 November 2013 • According to their previous roadmap, we should have seen SDTM 3.1.4 last July, and 3.1.5 roughly one year later • An updated roadmap has been published in late February • Stated long-term aim is to have two updates per year, à la MedDRA | BIAS Meeting | P. Nacci | 14 March 2014 | SDTM Implementation - Best Practices

29. CDISC Planned Activities and ReleasesWhat we can expect for the next year | BIAS Meeting | P. Nacci | 14 March 2014 | SDTM Implementation - Best Practices

30. Agenda The story so far Benefits of implementing SDTM Basic requirements Legacy data conversion: a missed opportunity? The way forward Conclusions

31. Conclusions • The old SAS server is still in service, but we are reducing the number of both business processes and users using it • We have plenty of exciting developments on our plate • CDR is a long-term project, which is evolving and will take years to reach a stable status • More and more activities are now moved completely to CDR • Safetysignaling using all available data for selected vaccines/products is now possible, and by the end of 2014 this should be possible for most of our major productscurrently on the market | BIAS Meeting | P. Nacci | 14 March 2014 | SDTM Implementation - Best Practices

32. Conclusions (2) • Using capabilities already built in SAS we should be able to create necessary documentation for the submission data (e.g., define.xml) with much less effort than in the past, and with a higher quality • In the meantime SDTM is moving again • Changes to existing processes have always a cost, and resources are not always a viable alternative to time • It will be crucial to understand how much change is introduced by each new version, and calculate the business impact in each company scenario • It might make sense to skip some ‘minor’ versions | BIAS Meeting | P. Nacci | 14 March 2014 | SDTM Implementation - Best Practices

33. Useful Links • CDISC: • http://www.cdisc.org • Technical Plan Project Schedule • SAS Clinical Standards Toolkit • http://support.sas.com/rnd/base/cdisc/cst/ • FDA/PhUSE Computational Science Symposium 2014 • http://www.phuse.eu/css | BIAS Meeting | P. Nacci | 14 March 2014 | SDTM Implementation - Best Practices

34. Useful Links (2) • FDA: • FDA-2014-D-0092- Study Data Technical Conformance Guide and Data Standards Catalog • FDA-2012-D-0097 - Guidance on Electronic Submissions: Standardized Study Data • FDA-2014-D-0085 - Guidance on Submissions in Electronic Format--Submissions under the Federal Food, Drug, and Cosmetic Act | BIAS Meeting | P. Nacci | 14 March 2014 | SDTM Implementation - Best Practices

35. Questions? | BIAS Meeting | P. Nacci | 14 March 2014 | SDTM Implementation - Best Practices