Diffusion-geometric maximally stable component detection in deformable shapes

1. Diffusion-geometricmaximally stable component detection in deformable shapes Diffusion-geometricmaximally stable component detection in deformable shapes Roee Litman, Alexander Bronstein, Michael Bronstein

2. In a nutshell… The Feature Approach for Images Deformable Shape Analysis MSER Maximally Stable ExtremalRegion Diffusion Geometry Shape MSER

3. Feature Approach in Images Feature based methods are the infrastructure laid in the base of many computer vision algorithms: • Content-based image retrieval • Video tracking • Panorama alignment • 3D reconstruction form stereo

4. Problem formulation • Find a semi-local feature detector • High repeatability • Invariance to isometric deformation • Robustness to noise, sampling, etc. • Add discriminative descriptor

5. The “what” Results

6. Visual Example

7. Visual Example

8. More Results (Taken from the TOSCA dataset) Horse regions + Human regions

9. Region Matching Query 1st, 2nd, 4th, 10th, and 15th matches

10. 3D Human Scans Taken from the SCAPE dataset

11. Scanned Region Matching Query 1st, 2nd, 4th, 10th, and 15th matches

12. Volume vs. Surface Original Volume & surface isometry Boundary isometry

13. Volumetric Shapes • Usually shapes are modeled as 2D boundary of a 3D shape. • Volumetric shape model better captures "natural" behavior of non-rigid deformations.(Raviv et-al) • Diffusion geometry terms can easily be applied to volumes • 2D Meshes can be voxelized

14. Volumetric Regions Taken from the SCAPE dataset

15. (The “how”) Methodology

16. Original MSER (Matas et-al)

17. MSER • Popular image blob detector • Near-linear complexity: • High repeatability [Mikolajczyk et al. 05] • Robust to affine transformation and illumination changes

18. MSER – In a nutshell • Threshold image at consecutive gray-levels • Search regions whose area stay nearly the same through a wide range of thresholds • Efficient detection of maximally stable regions requires construction of a component tree

19. MSER – In a nutshell

20. Algorithm overview

21. Algorithm overview • Represent as weighted graph • Component tree • Stable component detection

22. Algorithm overview • Represent as weighted graph

23. Image as weighted graph • An undirected graph can be created from an image, where: • Vertices are pixels • Edges by adjacency rule, e.g. 4-neiborhood

24. Weighting the graph In images • Gray-scale as vertex-weight • Color as edge-weight [Forssen] In Shapes • Curvature (not deformation invariant) • Diffusion Geometry

25. Weighting Option • For every point on the shape: • Calculate the prob. of a random walk to return to the same point. • Similar to Gaussian curvature • Intrinsic, i.e. – deformation invariant

26. Weight example Color-mapped Level-set animation

27. Diffusion Geometry • Analysis of diffusion (random walk) processes • Governed by the heat equation • Solution is heat distributionat point at time

28. Heat-Kernel • Given • Initial condition • Boundary condition, if these’s a boundary • Solve using: • i.e. - find the “heat-kernel”

29. Probabilistic Interpretation The probability density for a transition by random walk of length , from to

30. Spectral Interpretation • How to calculate ? • Heat kernel can be calculated directly from eigen-decomposition of the Laplacain • By spectral decomposition theorem:

31. Laplace-Beltrami Eigenfunctions

32. Deformation Invariance

33. Computational aspects • Shapes are discretized as triangular meshes • Can be expressed as undirected graph • Heat kernel & eigenfunctions are vectors • Discrete Laplace-Beltrami operator • Several weight schemes for • is usually discrete area elements

34. Computational aspects • In matrix notation • Solve eigendecomposition problem

35. Auto-diffusivity • Special case - • The chance of returning to after time • Related to Gaussian curvature by • Now we can attach scalar value to shapes!

36. Weight example Color-mapped Level-set animation

37. Algorithm overview • Represent as weighted graph • Component tree • Stable component detection

38. The Component Tree • Tree construction is a pre-process of stable region detection • Contains level-set hierarchy,i.e. nesting relations. • Constructed based on a weighted graph (vertex- or edge-weight) • Tree’s nodes are level-sets(of the graph’s cross-sections)

39. Tree Example

40. “Graphic” Example • A graph • Edge-weighted • 7 Cross-Section • 5 Cross-section • Two 5 level-sets(with altitude 4) • Every level-set has • Size (area) • Altitude (maximal weight) 4 7 8 9 8 1 4 1

41. Tree Construction 4 4 7 7 8 8 9 9 8 8 1 1 4 4 1 1

42. Algorithm overview • Represent as weighted graph • Component tree • Stable component detection

43. Detection Process • For every leaf component in the tree: • “Climb” the tree to its root, creating the sequence: • Calculate component stability • Local maxima of the sequenceare “Maximally stable components”

44. The Detils Performance

45. Benchmarking The Method • Method was tested on SHREC 2010 data-set: • 3 basic shapes (human, dog & horse) • 9 transformations, applied in 5 different strengths • 138 shapes in total Scale Original Deformation Holes Noise

46. Results

47. Quantitative Results • Vertex-wise correspondences were given • Regions were projected onto another shape, and overlap ratio was measured • Overlap ratio between a region and its projected counterpart is • Repeatability is the percent of regions with overlap above a threshold

48. Repeatability 65% at 0.75

49. Conclusion • Stable region detector for deformable shapes • Generic detection framework: • Vertex- and edge-weighted graph representation • Works on surface and/or volume data • Partial matching & retrieval potential • Tested quantitatively (on SHREC10)

50. Thank You Any Questions?