2D-3D Registration of X-Ray Fluoroscopy and CT Images (spinal column area)

1. 2D-3D Registration of X-Ray Fluoroscopy and CT Images(spinal column area) Petros Perselis ENGN2500: Medical Image Analysis Professor Benjamin Kimia Brown University Division of Engineering

2. Problem Statement - Approach • 2 datasets/modalities 3D CT volume (real spinal column data) 2D fluoroscopy images • Digitally Reconstructed Radiographs (DRRs) using Ray-Casting • 2D-3D Registration using Mutual Information (MI) and Gradient Ascent techniques • Sparsely Sampled Histogram Estimators

3. Background/Introduction Transformation T Objective Function g Optimization Summary Plan of Action

4. Background/Introduction (I) • Surgical operations: spinal column area • Delicate procedure • Image Guided Surgery • Pre-operative data (CT acquisitions) • Very high resolutions • BUT: slow, radiation • Intra-operative data (X-ray fluoroscopy) • Fast acquisitions • Less radiation • Simpler equipment • BUT: less details, noisy

5. Background/Introduction (II) • Registration: • Alignment of the 2 modalities • Combined advantages • Several methods exist • Challenges: • Different dimensionalities • Minimize computation time to fit operation

6. Transformation T (I) • Inputs: • 3D CT volume dataset • Biplanar 2D fluoroscopy images U(x) • 2 coordinate systems: imaging & world/data • Imaging environment is known • Transform imaging coordinates to data coordinates

7. Transformation T (II) (Sparse) Ray Casting to project 3D CT volume to DRRs V(T(x))

8. Objective Function g (I) • Measure of the alignment quality • Mutual Information (MI) to define g! • Intensity-based method • Information theory: knowledge of a random variable where and H is Shannon’s entropy

9. Objective Function g (II) Why Mutual Information? • Generality: does not assume linear relationship between random variables • Reduced computation time (combined with sparse X-ray casting and histogramic sampling)

10. Objective Function g (III) Goal optimize transformation T by maximizing g optimized registration

11. Optimization (I) • An optimization procedure aiming to maximize g • Stochastic Gradient-Ascent method • λ is the learning rate or step-size • We need to define !!!!

12. Optimization (II) • We have where and H is Shannon’s entropy • So, • And we have

13. Optimization (III) Through various approximations and simplifications, we conclude to: where and

14. Optimization (IV) So far, the stochastic nature of our approach is explained by: • Initial sparse ray casting (by random sampling) and construction of sparse histogramswith intensity bins • Approximation of statistical expectation terms with sample averages. Shannon’s entropy: • Noisy approximations for partial derivatives of probability densities by the use of finite differences calculated from the sparse histograms • Simplifications for the calculation of Ki,j

15. Summary Definition of objective function go by mutual information Iterative Optimization: 1. sparse ray casting to find DRRs 2. Calculate gradient of g 3. Update T based on gradient-ascent method Initial Transformation To Optimal registration when g reaches maximum

16. Plan of Action Weeks 1-3: studying of papers definition of the problem determination of our approach data gathering Week 4: writing a Matlab algorithm (or parts of it!), debugging, optimizing Week 5: testing on data, analyzing results conclusions (!?)

17. References • http://www.alain-pitiot.net/masters/images/registration.jpg • http://hdl.handle.net/1721.1/7078 • Zollei, L. Grimson, E. Norbash A. and Wells, W. “2D-3D Rigid Registration of X-Ray Fluoroscopy and CT Images Using Mutual Information and Sparsely Sampled Histogram Estimators” IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2001 vol. 2; p. II-696-II-703 • G.P. Penney, J. Weese, J.A. Little, P. Desmedt, D.L.G. Hill, D.J. Hawkes “A Comparison of Similarity Measures for Use in 2D-3DMedical Image Registration” IEEE Transactions on Medical Imaging 1998 vol 17 no. 4; p. 586-595

18. Thank you! QUESTIONS – COMMENTS – SUGGESTIONS