SNOMED CT - in Release Format 2 ‘RF2’

1. SNOMED CT- in Release Format 2 ‘RF2’ Implications for different constituencies Presented 22ndJuly 2013 Tom Seabury

2. Objectives • To share a sense of how SNOMED CT in RF2 can be fed into the terminology subsystems of an EHR system • To give an RF2 comparison to the RF1 approaches • for the same tasks

3. Sources A comprehensive exposition on RF2 is included in the SNOMED CT Technical Implementation Guide, as the section • Release Format 2 Update Guide • Information relating specifically to the UKTC RF2 distribution of its content is from the UKTC team

4. The impossible 40 minute Webinar? Technical subjects • New things, redundant things • Policy and tools • Major differences • ‘Concept Enumeration’ • Change log, history mechanism • RF2 release types • Content : Files : Folder structures • Implementation Schemas • Preferred terms, preferred FSNs

5. Webinar Topics Basics assumed knowledge from prior Webinars RF2 described & illustrated Policy and plans UKTC Plans RF2 compared to the pre-existing RF1 How do I … Tools

6. Assumed prior knowledge or experience RF1 • Awareness –or- • Experience of use –or- • Technical expertise SNOMED CT basics • The concept : descriptions : relationships scheme • The existence of an inactivation (‘history’) mechanism • Subsets (Refsets in RF2) • UK Extension content, its packing and distribution structures.

7. Unfamiliar words or concepts? During the webinar • please do ask for clarification if you encounter unfamiliar words or uses e.g. ‘Component’ has a specific meaning: - any of Concept, description, relationship, Refset e.g. ‘stated relationship’ - a relationship as stated by one of SNOMED CT’s authors, not one subsequently inferred from other relationships by some classifier tool. - Or seemingly contradictory statements

8. For Clinical users For: Clinical users of systems exploiting SNOMED CT RF2 has: No direct impacts – nothing will be immediately apparent

9. For informaticians (clinical or otherwise) For: Clinical / Informatician users • Those who configure systems exploiting SNOMED CT RF2 is: • Relevant to what tool are used • e.g. Refsets editor Vs. Subset editor • e.g. Detection of inactive content in templates, queries or results

10. For Developers and system developers For: Software & System Developers RF2: will need to be understood • Its format • The potential inherent in the available content • Which are the appropriate parts of SNOMED CT content to be correctly ingested and transformed • Any new Refset types (not needing revision to the RF2 standard) will need to be tracked and impact assessed.

11. For Content Developers For: UKTC Other developers of SNOMED CT content i.e. owners of their own SNOMED CT namespace RF2: • Will be conformant to RF2 in the development and maintenance of content and of metadata • Will be conformant to for the RF2 distribution of content • Can consider whether to embed RF2 techniques natively into authoring tools, or whether to rely on data transformations

12. Basics revisited

13. SNOMED CT Content, Release Formats SNOMED CT is a large set of reference data The UK Edition is distributed by the UKTC (via TRUD) Distribution formats are standardised In addition: local choices for ways to pack content are in use

14. SNOMED CT UK Edition The whole of the SNOMED CT UK Edition is distributed via TRUD A definition of ‘UK Edition’ • the International Release, • the UK Clinical Extension and • the UK Drug Extension

15. Data Content of SNOMED CT is distributed as • a collection of SNOMED CT data files of different types Files contain • the core SNOMED CT tables • sets of data components (in Refsets) • sets of metadata components (in Refsets) Refsets (RF2) or Subsets (RF1) : Sets of things (or more exactly) Collections of references to things e.g. a set of concepts cherry-picked from the whole of SNOMED CT ‘Tables’ and ‘Files’ are partly interchangeably used

16. SNOMED CT Content, Release Formats RF2 (and its predecessor RF1) are Standardised Release Formats for the content of SNOMED CT These are: • 99% Product and platform neutral • (exception: DOS eol characters) • Exclusively for use with SNOMED CT • Formalised in IHTSDO documentation as SNOMED CT Standards • Independent of the content which they distribute*

17. SNOMED CT Content, Release Formats An RF2 (full) release contains All of the past states of all the things which were ever in SNOMED CT UK Edition By contrast: an RF1 release contains All the things which were ever in SNOMED CT UK Edition, in their current state (RF2 Snapshot is similar to RF1, it has only current the status of any component. RF2 snapshot is however different from RF1 in many ways described later)

18. The UKTC use of RF1 and RF2 UK • UKTC have relied exclusively on RF1 until October 2012 • UK RF1 and RF2 will co-exist for no less than three years from October 2012, UK RF1 distribution is currently the definitive version. International • Deprecation of RF1 by IHTSDO is being considered, IHTSDO wish to distribute the international core content exclusively in RF2

19. RF2 described and illustrated

20. SNOMED CT content in RF2 RF2 is being used by UKTC to distribute: • Core content • Concepts • Descriptions • Relationships • Sets (formerly ‘subsets’ now as ‘Refsets’) • realm description Refset • ‘Non-Human’ concepts set • Cross-maps • e.g. to ICD-10

21. What structures and standards? RF2 standardises: • Data types • Attributes used, and their meaning • File types and naming (carrying numerous fragments of information) It is used to represent: • Core components • Reference sets (aka ‘Refsets’) • Essential functionality • (such as language specificity, historical status changes and associations

22. Concurrent to RF2 … • Introduction of ‘Module’ • and the Module Dependency Refset • ‘Active’ field • Each component in RF2 has an associated active field • values of true ('1') or false ('0') • Use to filter out inactive content where appropriate NB It is not always most appropriate to filter out inactive descriptions or concepts

23. Language of release formats ‘State Valid’ date stamped records ‘Refset’ Cf. Subset ‘Concept Enumeration’ self referencing ‘Delta, Snapshot, Full’ release types in RF2 ‘Module’, other new & existing metadata ‘Extensibility’ distribute anything

24. Log is ordered here in reverse chronological This is the first ever entry (SNOMED CT files have no defined ordering of rows) ‘State Valid’ illustrated And is subsequently inactivated Red text signified what has changed between entries For illustration, data is NOT colour coded Ownership changes so a new ‘Module’ association is recorded Modelling is improved and it becomes ‘fully defined’

25. Refset patterns (RF2) RF1 Subset Patterns UK map pattern UK Cross-maps ICD-10 

26. Reference Set names • The labels for Refsets can be more verbose • Addition of text to indicate the Refset type e.g. ‘Family history simple reference set’ reference set simple reference set Family history simple reference set (foundation metadata concept) Family history (foundation metadata concept) Implied purpose • Purpose clarified • in UK release documentation • on Subset register Explicit on formatting

27. RF2 Concept Enumerations Vs. RF1 arbitrary integers • RF2 - Concept enumerations are used across all release files. • uses concepts in a metadata hierarchy to represent an enumerated value set rather than using arbitrary integer (as in RF1) values • Take the SCTID data type

28. RF2 Concept Enumerations (and other Metadata) The metadata hierarchy

29. Zips, Files and folders

30. Zips, Files and folders This International release is not the baseline release, so Delta is legitimate to include

31. Choices of: Which of these am I likely to need? • Choice is between a current Snapshot and a current Full But what is mandated of UKTC? • The full view is required to support some SNOMED CT use cases but many requirements can be adequately met by providing access to a current Snapshot view. However: • ‘A SNOMED CT-enabled terminology server must be able to import data from a full release because this is the only Release Type that is required to be produced by all Extension developers’

32. Metadata values illustrated – Core ones

33. Metadata values illustrated - Module

34. Metadata values – carried over from RF1

35. UKTC distribution structures UKTC has always added further structure • beyond that mandated by the standard • e.g. • TRUD Packs and Subpacks • File:content strategy • e.g. which extension in what file • e.g. which sets in what file • No changes for RF2 introduction: replicated RF1 file and folder structures UK RF1 <> UK RF2

36. What does RF2 look like 197 files but Don’t Panic!

37. RF2 • Zipped structure • Unzipped structure • folder structures • file names • What is found where? • Data • Metadata • Continuity of access: Per RF1: • 2 full releases and each of the ‘incremental releases’ between them.

38. Familiar UK Folder names? UK Drug Extension

39. Illustration of Refset content & metadata

40. Policy and plans • UKTC Policy, IHTSDO policy • Transformabilty • Beyond the UK

41. UKTC policy – released data • UKTC enjoys some latitude in its cut-over from RF1 to RF2 • The planned period of concurrent running of RF1 plus RF2 (RF2 as tech preview status currently) will terminate in October 2015 • UK Edition in RF2 Status • Status: ‘Technical Preview’ • UK RF2 baseline release July 2013 • Scope – including UK cross-maps

42. UKTC policy – tooling • UKTC currently performs a conversion between RF1 and RF2 using tools and configuration data which is not itself distributed. • UKTC has no current plans to distribute these tools and configuration data (UKTC continues to author terminology content in tools which are not tied to any particular release format)

43. IHTSDO stated policy IHTSDO • RF2 ‘Developed in response to extensive feedback on’ RF1 • RF1 format was replaced by RF2 in January 2012, RF1 ‘is being maintained for a transitional period’ • (SNOMED CT®Technical Implementation Guide January 2013) UKTC policy reflects this, but is different

44. Transformability • Content is being transformed from RF1 to RF2 by UKTC • Content is being transformed from RF2 to RF1 by IHTSDO These transformations : • Require a fraction of prepared, different metadata for each format • Tables of equivalence for some metadata such as versioning of membership of Refsets and Subsets • For the UK Edition: are subject to a set of documented deviations provided by UKTC within the RF2 release note.

45. Forward Compatibility Today’s tools, tomorrows data (RF2 distributed) • UKTC Distribution in both RF1 and RF2 - UKTC Distribution in RF2 exclusively +/- metadata which is specific to and essential for RF1 +/- tools to allow you to generate RF1 (should you need to)

46. Back Compatibility Tomorrow’s tools, today’s data (RF1 distributed) • UKTC, UK Edition • RF2 metadata present (from RF2 files) • RF1 metadata present • Providing Back compatibility

47. SNOMED CT Release Formats – stability (Jan 2013) Stability ‘The RF2 format is likely to be stable for at least a five year period, without addition or deletion of fields’ Stable Extensibility The Refset mechanism permits (without change to the core standard) new Refset types to be used (extensibility)

48. Tools Over time, as RF2 becomes the primary distribution format in the UK, tools will be developed to enhance the ability to process data in this format more easily. This will include • Refset development • mapping tools • a concept editing environment

49. Similarities & differences RF2 and RF1

50. Contrast RF2 supports things which are unavailable from RF1: • Refset extensibility – a constrained set of novel types can be added RF2 supports things which differently available from RF1: • Component history Extensive documentation of the value and benefits is made by IHTSDO : http://www.snomed.org/guide/rf2value.pdf