Diffusion-geometric maximally stable component detection in deformable shapes

1. Diffusion-geometricmaximally stable component detection in deformable shapes Roee Litman, Dr. Alexander Bronstein, Dr. Michael Bronstein

2. The “Feature Approach”to Image Analysis • Video tracking • Panorama alignment • 3D reconstruction • Content-based image retrieval

3. Non-rigid Shapes

4. Problem formulation • Find a semi-local feature detector • High repeatability • Invariance to (non-stretching) deformation • Robustness to noise, sampling, etc. • Add informative descriptor

5. The Goal “Head+Arm” take a shape Detect (stable) regions “Head” “Arm” “Upper Body” “Leg” “Leg”

6. Deformation Invariant

7. More Results (Taken from the TOSCA dataset) Horse regions + Human regions

8. Region Description Distance = 0.44 Distance = 0.34 Distance = 0.02 Distance = 0.08 Distance= 0.17 Distance = 0.25

9. Region Matching Query 1st, 2nd, 4th, 10th, and 15th matches

10. 3D Human Scans Taken from the SCAPE dataset

11. Scanned Region Matching Query 1st, 2nd, 4th, 10th, and 15th matches

12. (The “how”) Methodology

13. In a nutshell… The Feature Approach for Images Deformable Shape Analysis Shape MSER MSER Maximally Stable ExtremalRegion Diffusion Geometry

14. Original MSER (Matas et-al)

15. MSER – In a nutshell • Threshold image at consecutive gray-levels • Search regions whose area stay nearly the same through a wide range of thresholds

16. MSER – In a nutshell

17. Algorithm overview

18. Algorithm overview • Represent as weighted graph • Component tree • Stable component detection

19. Algorithm overview • Represent as weighted graph

20. Weighting the graph In images • Illumination (Gray-scale) • Color (RGB) In Shapes • Mean Curvature (not deformation invariant) • Diffusion Geometry

21. Weighting Option • For every point on the shape: • Calculate the prob. of a random walk to return to the same point. • Similar to Gaussian curvature • Intrinsic - i.e. deformation invariant

22. Weight example Color-mapped Level-set animation

23. Diffusion Geometry • Analysis of diffusion (random walk) processes • Governed by the heat equation • Solution is heat distributionat point at time

24. Heat-Kernel • Given • Initial condition • Boundary condition, if these’s a boundary • Solve using: • i.e. - find the “heat-kernel”

25. Probabilistic Interpretation The probability density for a transition by random walk of length , from to

26. Spectral Interpretation • How to calculate ? • Heat kernel can be calculated directly from eigen-decomposition of the Laplacain • By spectral decomposition theorem:

27. Laplace-Beltrami Eigenfunctions

28. Deformation Invariance

29. Auto-diffusivity • Special case - • The chance of returning to after time • Related to Gaussian curvature by • Now we can attach scalar value to shapes!

30. Weight example Color-mapped Level-set animation

31. Algorithm overview • Represent as weighted graph • Component tree • Stable component detection

32. Performance

33. Benchmarking The Method • Method was tested on SHREC 2010 data-set: • 3 basic shapes (human, dog & horse) • 9 transformations, applied in 5 different strengths • 138 shapes in total Scale Original Deformation Holes Noise

34. Results

35. Quantitative Results • Regions were projected onto “original” shape,and overlap ratio was measured • Vertex-wise correspondences were given • Overlap ratio between a region and its projected counterpart is • Repeatability is the percent of regions with overlap above a threshold

36. Repeatability 65% at 0.75

37. Conclusion • Stable region detector for deformable shapes • Generic detection framework • Vertex- and edge-weighted graph representation • Surface and volume data • Partial matching & retrieval potential • Tested quantitatively (on SHREC10)

38. Thank You Any Questions?