Integrating the Healthcare Enterprise

1. Integrating the Healthcare Enterprise Todd Kantchev TK Medical Systems Jerold Wallis Washington University Image Fusion

2. Image Fusion Scenario: I want to view two studies (e.g. a PET and a CT) overlayed on the screen and I want to look at it again later, maybe even on another screen 2

3. Your Problems with Image Fusion • “I registered these before. Why do I have to do it again?” • Registrations not transferable between systems • “Why can’t I see it on another workstation?” • Presentations not transferable between systems • “Why can I only do PET / CT?” • No general purpose registration • Impact • time lost to repeated work; delayed care • inability to access needed views

4. How should it work? I scan on a dual system or several systems Some datasets are pre-registered The tech does additional registrations or tweaks the pre-registered sets On my workstation, I adjust the blending and read the images Later, on another workstation, I retrieve & view perhaps tweak the registration perhaps tweak the blending perhaps register additional datasets 4

5. IHE Solution (FUS) – Image Fusion Record Registration between any datasets Record Blending / Presentation parameters Retrieve & View on any compliant workstation 5

6. SRO Spatial Registration Object • (Not required for already-registered PET-CT data from a hybrid scanner ) Image 1 Image 2 Transformation

7. Spatial Registration Object • Encodes transformation to align images in a common registered space • Identifies the registered space • References the images to be aligned • Not required for already-registered PET-CT data from a hybrid scanner

8. SRO BPS Blending Presentation State Image 1 Transformation Window – 1 Image 2 Window – 2 Color Table Transparency

9. Blending Presentation Object • References one set of underlying images (e.g. CT) • References one set of superimposed images (e.g. PET) • References (optionally) a Registration Object to align both image sets before blending. • Encodes the window level for each set • Encodes the blending factor (0 to 100%) of the overlay transparency

10. Essential Clinical Features • MPR • Fusion • Side-by-side • Registered

11. Image Fusion Actors and Transactions Acquisition Modality Evidence Creator [RAD 8] Modality Images Stored ↓ [RAD 57] Blending Presentation States Stored ↓ [RAD 56] Spatial Registrations Stored ↓[RAD 10] Storage Commitment ↓ ↓ [RAD 18] Creator Images Stored ↓ [RAD 57] Blending Presentation States Stored ↓ [RAD 56] Spatial Registrations Stored ↓ [RAD 10] Storage Commitment Image Manager Image Archive ↓ [RAD 14] Query Images ↓ [RAD 16] Retrieve Images ↓ [RAD 15] Query Presentation States ↓ [RAD 17] Retrieve Presentation States ↓ [RAD 58] Retrieve Spatial Registrations Image Display Vol 1-20.1 Actors and Transactions in Image Fusion Profile

12. Image Fusion Actors and Transactions Evidence Creator Acquisition Modality [RAD 8] Modality Images Stored ↓ [RAD 57] Blending Presentation States Stored ↓ [RAD 56] Spatial Registrations Stored ↓[RAD 10] Storage Commitment ↓ ↓ [RAD 18] Creator Images Stored ↓ [RAD 57] Blending Presentation States Stored ↓ [RAD 56] Spatial Registrations Stored ↓ [RAD 10] Storage Commitment Image Manager Image Archive ↓ [RAD 14] Query Images ↓ [RAD 16] Retrieve Images ↓ [RAD 15] Query Presentation States ↓ [RAD 17] Retrieve Presentation States ↓ [RAD 58] Retrieve Spatial Registrations Image Display

13. Load CT and PET images Align images (automatically or manually) and fuse with desired blending factor Store the alignment, including Registration object Blending State (defining the blending factor, window levels and referencing all other objects) Storage can be on PACS or disk. Images do not need to be stored if already stored. Fusion- The Creating Process CT PET Blending State Registration Object

14. Acquisition Modalities • Hybrid Modalities are required to store Blending Softcopy Presentation State for the datasets, which are in the same Frame of Reference. Typically, the Referenced Spatial Registration Sequence is absent since no transformation needs to be applied to register the referenced images before display. • All modalities on the IHE TF are now explicitly required, when creating image objects, not to use the same Frame of Reference for multiple image datasets unless all the images are in the same Reference Coordinate System. 4.57.4.1.2 Message Semantics – BSPS Stored

15. TEST • FUS_MOD_STORE: Modality Stores Images and BSPS to Central Archive • Special Instructions • Acquistion Modalities should complete this test on the first day of the Connectathon to enable Evidence Creators, Image Display and Image Archives to complete their tests that depend on this one. • No formal evaluation ? Validate BSPS?

16. Image Fusion Actors and Transactions Evidence Creator Acquisition Modality [RAD 8] Modality Images Stored ↓ [RAD 57] Blending Presentation States Stored ↓ [RAD 56] Spatial Registrations Stored ↓[RAD 10] Storage Commitment ↓ ↓ [RAD 18] Creator Images Stored ↓ [RAD 57] Blending Presentation States Stored ↓ [RAD 56] Spatial Registrations Stored ↓ [RAD 10] Storage Commitment Image Manager Image Archive ↓ [RAD 14] Query Images ↓ [RAD 16] Retrieve Images ↓ [RAD 15] Query Presentation States ↓ [RAD 17] Retrieve Presentation States ↓ [RAD 58] Retrieve Spatial Registrations Image Display

17. Evidence Creators • Create and store Blending Softcopy Presentation objects as required • Referenced Spatial Registration Sequence (0070, 0404) references the Spatial Registration object, unless spatial registration is not required • The Presentation State Relationship Macro in the Blending Sequence (0070, 0402) lists all the images or frames to which blending is to be applied • Encodes presentation (blending, hue, windowing etc.) • Create and store Spatial Registration Objects as required • Contains one or more Registration Sequences, each defining the transformation of the corresponding Original Dataset into the Registered Space. • Stored in the most recent of the Studies containing the registered images and in a different series from images and presentation states. • Optionally create derived (re-sampled) images. 4.56.4.1.2 Message Semantics – Spatial Registration Stored 4.57.4.1.2 Message Semantics – BSPS Stored See also 4.18.4.1.2.5 for storage of Derived Images

18. Test: FUS_MOD_EC • FUS_MOD_EC: Evidence Creator performs registration and blending on Modality image series • Images will be on the Connectathon CENTRAL_ARCHIVE after Modalities complete their FUS_MOD_STORE test. • This is a test between Modality and Evidence Creator actors

19. Demonstrate that Evid. Creator • …register image datasets which do not share a Frame of Reference, using translations, rotation & scaling, as required • …(recommended) that it notes the method of registration within the Spatial Registration object. • …it can set the contrast/window settings for the underlying grayscale dataset, and … it can set the windowing and color map for the overlay dataset • … it can set the transparency for the overlay dataset

20. FUS_MOD_EC • Create several examples of the presentation state, with different registration and window/blending settings (eg, different transparency or color maps) • Store them to the archive for use by Displays.

21. Image Fusion Actors and Transactions Evidence Creator Acquisition Modality [RAD 8] Modality Images Stored ↓ [RAD 57] Blending Presentation States Stored ↓ [RAD 56] Spatial Registrations Stored ↓[RAD 10] Storage Commitment ↓ ↓ [RAD 18] Creator Images Stored ↓ [RAD 57] Blending Presentation States Stored ↓ [RAD 56] Spatial Registrations Stored ↓ [RAD 10] Storage Commitment Image Manager Image Archive ↓ [RAD 14] Query Images ↓ [RAD 16] Retrieve Images ↓ [RAD 15] Query Presentation States ↓ [RAD 17] Retrieve Presentation States ↓ [RAD 58] Retrieve Spatial Registrations Image Display

22. Image Manager/Archive • Able to store, query and retrieve Blending Softcopy Presentation States, supporting all required keys, already defined in the TF. Note that SOP Class UID (a required matching key for the SCP) is a useful way to limit responses to queries for Blending Softcopy PS. • Able to retrieve Spatial Registration Objects. • Retrieval of images must be supported at Series Level. 4.15.4.1.2 Message Semantics- Query Softcopy PS

23. Image Fusion Actors and Transactions Evidence Creator Acquisition Modality [RAD 8] Modality Images Stored ↓ [RAD 57] Blending Presentation States Stored ↓ [RAD 56] Spatial Registrations Stored ↓[RAD 10] Storage Commitment ↓ ↓ [RAD 18] Creator Images Stored ↓ [RAD 57] Blending Presentation States Stored ↓ [RAD 56] Spatial Registrations Stored ↓ [RAD 10] Storage Commitment Image Manager Image Archive ↓ [RAD 14] Query Images ↓ [RAD 16] Retrieve Images ↓ [RAD 15] Query Presentation States ↓ [RAD 17] Retrieve Presentation States ↓ [RAD 58] Retrieve Spatial Registrations Image Display

24. Image Display • Able to retrieve, read and apply to images Blending Softcopy Presentation States and, if referenced, Spatial Registration Objects as required • Able to retrieve the referenced images, if not already present locally, and perform image registration as required if necessary • Display images, supporting specific display features (fusion, side by side, MPR, adjustment of blending, windowing- aka NM, navigation through spatial and time dimensions, display content of specific attributes etc.) 4.17.4.2.3 Retrieve/View Presentation States. 4.58.4.1.2 Retrieve Spatial Registrations

25. Essential Clinical Features • MPR • Fusion • Side-by-side • Registered

26. Blending PresentationOverlay Transparency 20% 50% 80% 50% 50% 100% 50%

27. Example- Side by Side • Side-by-side can be up/down (PET/CT) • Side-by-side can omit fusion image when displayed in this format • Recommended – MIP image capability (upper left)

28. Example- Side by Side • Side-by-side can be up/down, can be multiple levels (eg 3 levels as shown here) • Side-by-side can include fusion image (preferred)

29. Example- Side by Side • Side-by-side can include multiple CT window settings (as shown here) • Side-by-side can include fusion image (preferred)

30. Example- Side by Side • PET-only image (upper right) can be pure BW (preferred as on prior slide, but not mentioned in profile), but can use same LUT as fused image as shown here

31. Example- MPR • MPR screen must be able to show 3 planes • MPR screen need not show side-by-side as well, but can do so (here with pure CT image at upper left, and fused below)

32. Example- MPR • Must be able to adjust blending (transparency) • Same screen adjusted to be mostly CT

33. Example- MPR • Must be able to adjust blending (transparency) • Same screen adjusted to be mostly PET

34. Example- MPR • Different vendor example for MPR fused

35. Fusion controls • Must have controls for • Windowing CT (using standard center/width controls) • Windowing PET/NM (using separate upper & lower level controls , not center/width controls • Adjusting transparency • Navigating volume in three planes • Changing color LUT table for overlay is recommended but not required

36. Example- Controls • Must have controls for • Windowing CT (using standard center/width controls) • Windowing PET/NM (using separate upper & lower level controls , not center/width controls • Adjusting transparency • Navigating volume in three planes • Changing color LUT table for overlay is recommended but not required

37. Example- Controls • Must have controls for • Windowing CT (using standard center/width controls) • Windowing PET/NM (using separate upper & lower level controls , not center/width controls • Adjusting transparency • Navigating volume in three planes • Changing color LUT table for overlay is recommended but not required

38. Example- Controls • Must have controls for • Windowing CT (using standard center/width controls) • Windowing PET/NM (using separate upper & lower level controls , not center/width controls • Adjusting transparency • Navigating volume in three planes • Changing color LUT table for overlay is recommended but not required

39. Selecting Studies • Users must be able to select the softcopy presentation state (BSPS) • More than one BSPS may be present • If not already present, any images or spatial registration objects referenced in the Presentation State shall be retrieved by the Image Display. (For clarification on this, see closed issue 19 and section 4.17.4.2.3 )

40. Full Profile on IHE Site • http://www.ihe.net/Technical_Framework/index.cfm#radiology • http://www.ihe.net/Technical_Framework/index.cfm#radiology • Open link on page above for FUS profile