Automated Medical Image Registration using Global and Conditional Mutual Information Dirk Loeckx Frederik Maes, Dirk Vandermeulen, Paul Suetens
Medical Imaging Research Center Medical Image Computing Group of BiomedicalSciences Group of Science, Engineeringand Technology Radiology, Nuclear Medicine, Cardiology, Radiotherapy Processing of Speech & Images
Image Registration Find geometrical relationship between images
Why? • Wealth of images • Registration • Integrate information from different images • Different modality, time, patient, pose, contrast • Automatic ↔ implicit • Clinical workflow • Fully digital • Acquisition, transmission, storage, retrieval, reading • Processing time, visualisation • Validation
Input data Similarity Measure Transformation Penalties R Smoothness Volume Rigidity F Optimizer Deformed Image Visualization Multiresolution Analytic Derivatives LBFGSB (quasi Newton) F’ Building Blocks
Similarity Measure Sum of Squared Differences (SSD) Constant Relation Cross Correlation(CC) Linear Relation Mutual Information(MI) Statistical Relation
Mutual Information • ‘Information’ Rcarries about F and vice versa • Based on joint histogram • Combine intensity R(x) with F(g(x;m))in discrete bins • If I know p(r), how good can I predict p(f)? • Joint and marginal probabilityp(r;m), p(f;m), p(r,f;m) • F(g(x;m))unknown (digital images, partial volume effect) • No spatial information!
Mutual Information + – = : Reference image (histogram) : Floating image (histogram) : Entropy
Mutual Information • Excellent results for rigid registration • Only translation/rotation (6 parameters) • Parameters influence whole image • Sometimes works for nonrigid registration • 100’s to 1000’s of parameters • Local influence
Rigid CT/MR F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens, “Multimodality image registration by maximization of mutual information,” IEEE Trans. Med. Imag., vol. 16, no. 2, pp. 187–198, Apr 1997.
Nonrigid Cardiac US • Left ventricle • Manual delineation • Triangulation • Propagate delineation • Mutual information image registration • Backward and forward • Constant connectivity • 4D path of each vector • Visual validation: ok
Subtraction CT angiography • 3D acquisitions of region of interest • Without / with contrast • Registration • Nonrigid mutual information • Compensate for movement artefacts • Subtraction • Only contrast remains • Clear view of vasculature
MI for nonrigid registration • Ongoing field of research • Global Histogram • Multimodal registration • Minimise minor in favour of major peaks • Reduce smaller image details • Bias field • Register bias fields, not image features (also rigid) • Local Histogram (image subdivision) • Limited ‘hits’: statistical power? • Solution: overlapping subregions
- Reference position Floating position Tensor-product B-splines Rectangular mesh Displacement vectors Weighted sum B-splines Multiresolution, limited span, analytical derivative Transformation model
Local Mutual Information • Overlapping subregions • : Spatial label, spatial bins • Multidimensional mutual information? • Total correlation, 3 channel regional MI • ‘Amount of redundancy in a set of variables’ • Conditional MI • ‘MI between R and F when X is known’ • …
Conditional MI (cMI) – = + • Locally, if I know p(r), • can I better predict p(f)?
In theory R F Global MI: whole image local optimum Conditional MI: central region global optimum
200 2D image pairs ‘CT’, ‘MR’ 256x256 pixels I = 0, 200, 400, noise s =50 Mesh spacing 32 voxels 32 bins, PW, PV Initial transformation m uniform, < 30 pixels Validation Intensity difference Warping index ROI: 10% outside polygon Multimodal registration CT (original) MR (warped) PV, global MI PW, global MI PV, conditional MI PW,conditional MI
Multimodal registration CT (original) MR (warped) PV, global MI PW, global MI PV, conditional MI PW,conditional MI
200 2D image pairs Lena image 8 bit, 256x256 pixels Floating: Bias field 2nd degree multiplicative Initial transformation m uniform, < 30 pixels Validation Intensity difference Warping index Bias field registration CT (original) MR (warped) PV, global MI PW, global MI PV, conditional MI PW,conditional MI
Bias field registration CT (original) MR (warped) PV, global MI PW, global MI PV, conditional MI PW,conditional MI
Conclusion • Small structures • Global joint entropy has two local optima • Bias towards minimal floating entropy • Local joint entropy has single optimum • Combination of joint and marginal entropy • Bias fields • Global MI • Combines contributions from all over the image • Bias field: widening of tissue histogram peaks • Local MI • Local histograms only • Less widening
Conclusion • Mesh size? • Smaller mesh = fewer voxels/bin = less statistical power • Compensated by overlapping B-spline windows (?) • 2D: 9216 voxels/bin (~ 3D: 7x7x7 voxels) • Calculation time • 10x for 2D (200s), 20x for 3D (days) • Other conditional similarity measures • Conditional Cross Correlation