A multispectral image enhancement approach to visualize tissue structures

1. A multispectral image enhancement approach to visualize tissue structures Pinky A. Bautista1,Tokiya Abe1, Yukako Yagi1, John Gilbertson1, Masahiro Yamaguchi2, and Nagaaki Ohyama2 1 Massachusetts General Hospital 2 Technology Institute of Technology

2. Multispectral Imaging (MSI) • Originally developed for space-based imaging • Multiple grey-level images are captured at different wavelengths • Allows extraction of additional information which the human eye fails to capture. Filter sensitivities RGB imaging Mutispectral imaging MSI allows greater flexibility for image analysis as compared to RGB imaging N Narrowband filters 3 broadband filters 3 grey-level images N>3 grey-level images

3. Objectives • To digitally enhance an H&E stained multispectral image such that collagen fiber can easily be differentiated from the rest of the eosin stained tissue components. • Show the capability of multispectral imaging to differentiate tissue structures with minute colorimetric difference.

4. Multispectral Microscope Imaging system • Olympus BX-62 optical microscope controlled by a PC • 16 interference filters • 2kx2k pixel CCD camera *Used in the experiment

5. Controls the color of enhanced areas Enhancement Method Spectral residual error Original spectral transmittance at location x,y (16-band) W NxN weighting factor Matrix, i.e. N=16 Enhanced version estimated spectral transmittance using M (M<N) KL vectors derived from the transmittance data of the selected tissue components Reference: Masanori Mitsui, Yuri Murakami, Takashi Obi, et.al, “ Color Enhancement in Multispectral Image Using the Karhunen-Loeve Transform,” Optical Review Vol.12, no.2, pp.60-75, 2005

6. Experiment 1. Training Phase 2. Testing Phase • Collection of 16-band transmittance spectra samples of the identified H&E stained tissue components • Perform multispectral enhancement on 16-band images using the M-KL vectors derived in the training phase • Derivation of the KL vectors • Transform the multispectral enhanced image into its equivalent RGB format for visualization • Identification of the appropriate number of KL vectors, i.e. M-KL vectors Examine the spectral residual error characteristics of the different tissue components

7. Derivation of KL vectors Subject for enhancement Training data Transmittance spectra of the different tissue components fiber RGB format of the 16-band MS image of a Heart tissue Not Subject for enhancement Nucleus Cytoplasm RBCs, etc. KL vectors were derived from the transmittance of these tissue components

8. Tissue components transmittance spectra structures found in white areas Each tissue component is represented with 200 transmittance spectra samples.

9. Spectral Residual Error Appropriate number of KL vectors was investigated…… structures found in white areas The spectral residual error for fiber peaks at band 8 5-KL vectors were found to produce distinct peaks on the spectral residual error of collagen fiber

10. Result (heart tissue) H&E stained Digitally enhanced Striated muscle Striated Muscle Collagen fiber Collagen fiber • 2kx2k pixels Striated muscle and Collagen fiber which are both stained with Eosin in an H&E stained slide are impressed with different shades of color when digitally enhanced • 20x magnification

11. Results Serial Section H&E stained Digitally enhanced MT stained reference Tissue areas highlighted in the digitally enhanced image correspond to areas emphasized by the MT stain

12. Result (Magnified) Original H&E stained Enhanced H&E stained image MT stained not clearly differentiated differentiated differentiated reference Tissue structures with minute color difference is differentiated using Multispectral information

13. RGB and Multispectral Serial Section MT stained image Original H&E stained image Enhanced using RGBinformation Enhanced using Multispectral information

14. RGB and Multispectral Enhanced using RGBinformation Enhanced using Multispectral information Original H&E stained image Not clearly differentiated Clearly differentiated

15. Spectral transmittance There is a slight difference in the spectral configurations between the labeled fiber1 and fiber2 areas

16. Future work Conclusion • With multispectral imaging it is possible to differentiate tissue structures with minute colorimetric difference • The current enhancement scheme makes it possible to differentiate tissue structures that are less likely differentiated with RGB imaging • Work with more tissue images to validate the current result • Investigate further the meaning of spectral residual error in relation to tissue differentiation • Investigate possible application of the residual error configurations to select important bands to classify/segment specific tissue structures

17. THANK YOU…. We thank CAP foundation for making it possible for us to attend this conference.

18. Weighting matrix Variation Spectral enhancement H&E stained Digitally enhanced Color of target areas can be varied by manipulating the weighting matrix W

19. Results Serial Section H&E stained Digitally enhanced MT stained reference Tissue areas highlighted in the digitally enhanced image correspond to areas emphasized by the MT stain

20. Result (kidney tissue) H&E stained Digitally enhanced Training data were extracted from another MS image of a kidney tissue; training and test images belong to the same slide

21. Result (kidney tissue) H&E stained Digitally enhanced MT stained