1. Content-Adaptive 3D Mesh Modeling for Representation of Volumetric Images Jovan G. Brankov, Yongyi Yang, and Miles N. Wernick Illinois Institute of Technology Research supported by Whitaker Foundation and NIH/NHLBI HL65425 www.ipl.iit.edu

2. Motivation: Our project in 4D reconstruction using mesh modeling • We are exploring mesh modeling for image representation and 4D reconstruction with elastic motion tracking • Applied in field of nuclear medicine : • Fast and accurate mesh generation (2D1) • Mesh model image tomographic reconstruction (2D2,3, 3D4) • Myocardium motion tracking and 4D post-processing (2D+time)5 • Dual-modality image reconstruction using mesh modeling6 • Improvements in reconstruction performance: • SNR 2,3,4,5 • Defect detectability 3 • Reconstruction time 2,3 • Memory requirement 2,3 • This paper: Content-Adaptive 3D Mesh Modeling for Representation of Volumetric Images 1. Y. Yang, J. G. Brankov, and M. N. Wernick, IEEE ICIP 2001 2. J. G. Brankov, Y. Yang , and M. N. Wernick et al. IEEE ICIP 2001 3. J.G. Brankov, at. al. Fully 3D Image Recon., 2001 4. Y. Yang, J. G. Brankov, and M. N. Wernick IEEE ICIP 2002 5. J. G. Brankov, et al. Fully 3D Image Recon., 20016. J. G. Brankov, et al. IEEE ISBI 2002 www.ipl.iit.edu

3. Ultimate goal: 4D reconstruction with motion tracking • Tracking tissue elements and utilize the temporal correlation to reduce noise; • Non pixel representation. Mesh model with motion tracking Multichannel* (motion) *N. P. Galatsanos, M.N. Wernick, and A.K. Katsaggelos, “Multichannel Image Recovery,” in Handbook of Image and Video Processing, A. Bovik, Ed., San Diego: Academic Press, pp. 155-168, 2000. www.ipl.iit.edu

4. Content-adaptive mesh modeling (2D example) • Non-uniform sampling replaces conventional pixels • Put more samples (nodes) in high-frequency image regions • Partition image domain into non-overlapping patches, called mesh elements • Image function interpolated over each element from its nodal values = mesh node = interpolation basis function N= number of mesh nodes Mesh element www.ipl.iit.edu

5. Potential advantages • Mesh modeling is a compression algorithm, thus • reduce number of unknown to be estimated • shorter computation time • lower memory requirement • Built-in spatially-adaptive smoothing • Natural framework for 4D reconstruction with motion tracking (deformable mesh) (our goal) 2D examples are essayer to be visualized; therefore for intuitive explanation 2D examples are given www.ipl.iit.edu

6. 2D: Theoretical basis* • Theorem: If the image is twice-differentiable, then the approximation error of linear interpolation over a triangular element is bounded as follows: where h = the length of the longest side of mesh element = largest magnitude of the second directional derivatives over the element • Implication: Achieve relatively uniform error bound throughout image by making density of mesh elements proportional to maximum magnitude of 2nd directional derivatives *Y. Yang, M.N. Wernick, and J.G. Brankov, Proceedings of IEEE ICIP 2001. www.ipl.iit.edu

7. 2D: Mesh generation method Original Step 1: Feature Map Step 2: Nodal positions - Halftoning Step 3: Mesh structure - triangulation Step 4: Interpolation www.ipl.iit.edu

8. 2D: Basis for new method: Halftoning Distribute ink dots adaptively, proportional to the local image intensity. Floyd-Steinberg algorithm: numerically efficient; single pass. www.ipl.iit.edu

9. e(p) 7/16 3/16 5/16 1/16 Halftoning by error diffusion* • Starting from the 1st pixel, proceed in a raster scanning order • At each pixel x • mesh nodes placed at pixels where n(p)=1 • T controls the number of nodes *R. Floyd and L. Steinberg, SID Intl. Sym. Digest of Technical Papers, 1975. www.ipl.iit.edu

10. 2D: Delaunay triangulation Connects a given set of mesh nodes in such a way that the circle circumscribing any triangular element contains only the nodal points belonging to that triangle • yields a well-structured mesh • reasonable computational cost • avoids producing excessively elongated elements, thereby further reducing the error bound www.ipl.iit.edu

11. 3D: Theoretical basis* • Theorem: If the function is twice-differentiable, then the approximation error of linear interpolation over a tetrahedron element is bounded as follows: where h = the length of the longest side of mesh element = largest magnitude of the second directional derivatives over the element • Implication: Achieve relatively uniform error bound throughout image by making density of mesh elements proportional to maximum magnitude of 2nd directional derivatives raised to 1.5 power. www.ipl.iit.edu

12. 3D: Mesh generation • Extract feature map (second directional derivative) • Applied modified error diffusion • The diffusion coefficients are chosen to be inversely proportional to its distance to the current voxel and directly proportional to the function 2nd derivative in the direction of diffusion. • 3D Delaunay triangulation* • Interpolate volume form mesh representation *The National Science and Technology Research Center for Computation and Visualization of Geometric Structures - University of Minnesota www.ipl.iit.edu

13. Conventional mesh approaches tested Two Dimension (2D) Quadtree mesh generation1 • Mesh elements with large interpolation errors are successively sub-divided into smaller elements Three Dimension (3D) Octree mesh generation • Extension of quadtree 1. Y. Wang and O. Lee, IEEE Trans. Circuits Syst. Video Tech., vol. 6, pp. 636 -646, 1996 www.ipl.iit.edu

14. Mesh nodal position placement Octtree method Proposed method www.ipl.iit.edu

15. Inside of myocardium Proposed method Octtree method www.ipl.iit.edu

16. Proposed method : Least square fitted images www.ipl.iit.edu

17. Octtree method: Least square fitted images www.ipl.iit.edu

18. Conclusion • Proposed method is a fast an accurate content adaptive mesh procedure; • Produces better results than classical (octtree) method; • New building block to reach 4D motion compensated reconstruction by a way of mesh modeling; www.ipl.iit.edu