Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence

1. Coarse-to-Fine CombinatorialMatching for Dense Isometric Shape Correspondence Yusuf Sahillioğlu and Yücel Yemez Computer Eng. Dept., Koç University, Istanbul, Turkey

2. Problem Definition & Apps Goal: Find a mapping between two isometric shapes • Shape interpolation • Attribute transfer • Shape registration • Time-varying recon. • Shape matching • Statistical shape analysis Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

3. Contributions • Avoid embedding • C2F joint sampling of evenly-spaced salient vertices Euclidean embedding Non-Euclidean embedding geodesic curvature integral • O(NlogN)time complexity for dense correspondence Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

4. Isometry • Our method is purely isometric • Intrinsic global property • Similar shapes have similar metric structures • Metric: geodesic distance Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

5. Isometric Distortion • Given,measure its isometric distortion: in the most general setting. : normalized geodesic distance b/w two vertices. Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

6. Isometric Distortion g g g g g g g in action: g average for . Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

7. Minimizing Isometric Distortion • N = |S| = |T|for perfectly isometric shapes. • N!different mappings; intractable. • Solution: Patch-by-patch matching to reduce search space. • Optimal mapping maps nearby vertices in source to • nearby vertices in target. • Recursively subdivide matched patches into smaller patches (C2F sampling) to be matched (combinatorial search). Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

8. Coarse-to-Fine Sampling • : set of base vertices sampled fromat level . • Sampling radii s.t. fork=0,1,..,K. • at level defines patch : all vertices within a distance from the base . greens inherited from levelk−1 blues are all vertices ( ) blacks + greens = patches being defined ( ) Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

9. Correspondence Algorithm • Correspondence at level k is obtained in two steps: • Match level k bases inside the patch pairs matched at level k−1. • Merge patch-based local correspondences into one global correspondence over whole surface. Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

10. Patch-based Matching ( ) • Ensure base vertices fall into each patch to allow combinatorial matching. • Patch radius to select for such an : , area of the largest patch at levelk−1. (enlarge a bit to cover whites) M=5samples with circular patches to cover blue area Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

11. Patch-based Matching ( ) • Combinatorial matching greens inherited from level k−1 blacks + greens = Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

12. Correspondence Merging ( ) • Merge patch-to-patch correspondences into one global correspondence that covers the whole surface. Multi-graph  single graph. Also, disovalues made available. Trim matches with diso> 2Diso, i.e., outliers. 1st pass over source samples to keep only one match per sample, the one with the min diso. 2nd pass over target samples to assign one match per isolated sample, the one with the min diso. Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

13. Insight to the Algorithm • Conditions for the algorithm to work correctly • High-resolution sampling on two perfectly isometric surfaces • Evenly-spaced sampling s.t. every vertex is in at least one patch • Distortion is a slowly changing convex function around optimum • One optimal solution (no symmetric flips) • Optimal mapping assigns si to tjwhich is as nearest to the ground-truth ti as possible • Inclusion assertion is then expected to apply: Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

14. Inclusion assertion (demonstration) Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

15. Computational Complexity • Saliency sorting • C2F sampling Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

16. Computational Complexity • Patch-based combinatorial matching • Merging Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

17. Computational Complexity • Overall Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

18. Experimental Results • Details captured, smooth flow • Many-to-one 6Kvs.16K red line: the worst match w.r.t. isometric distortion • Two meshes at different resolutions Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

19. Experimental Results red line: the worst match w.r.t. isometric distortion Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

20. Experimental Results • for four more pairs: red line: the worst match w.r.t. isometric distortion green line: the worst match w.r.t. ground-truth distortion Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

21. Experimental Results • Comparisons Nonrigid world dataset GMDS O(N2logN) [Bronstein et al.] Spectral O(N2logN) [Jain et al.] Our method O(NlogN) Our method O(NlogN) Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

22. Future Work • A solution for symmetric flips due to initial coarse sampling: • Symmetric flip issue • Purely isometry-based methods naturally fail at symmetric inputs • Not intrinsically symmetric  only one optimal solution • Our method may still occasionally fail to find the optimum due to initial coarse sampling • Solution suggested Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

23. Conclusion • Computationally efficient C2F dense isometric shape correspondence algorithm (O(NlogN)). • Isometric distortion minimized in the original 3D Euclidean space wherein isometry is defined. • Accurate for isometric and nearly isometric pairs. • Different levels of detail thanks to the C2F joint sampling. • No restriction on topology. • Symmetric flips may occasionally occur due to initial coarse sampling (but can be healed as proposed). Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

24. People Yusuf, PhD student Assoc. Prof. Yücel Yemez, supervisor