Alignment and Matching

1. Alignment and Matching Thomas Funkhouser and Michael Kazhdan Princeton University

2. Translation Scale Rotation Challenge • The shape of model does not changewhen the model is translated, scaled,or rotated =

3. Outline • Matching • Alignment • Exhaustive Search • Invariance • Normalization • Part vs. Whole • Conclusion

4. Exhaustive Search • Search for the best aligning transformation: • Compare at all alignments • Match at the alignment for which models are closest Exhaustive search for optimal rotation

5. Exhaustive Search • Search for the best aligning transformation: • Compare at all alignments • Match at the alignment for which models are closest

6. Exhaustive Search • Search for the best aligning transformation: • Use signal processing for efficient correlation • Represent model at many different transformations • Properties: • Gives the correct answer • Is hard to do efficiently

7. Outline • Matching • Alignment • Exhaustive Search • Invariance • Normalization • Part vs. Whole • Conclusion

8. Invariance • Represent a model with information that is independent of the transformation: • Extended Gaussian Image, Horn: Translation invariant • Shells Histograms, Ankerst: Rotation invariant • D2 Shape Distributions, Osada: Translation/Rotation invariant Shells Histogram D2 Distribution EGI

9. Invariance • Represent a model with information that is independent of the transformation • Power spectrum representation • Fourier Transform for translation and 2D rotations • Spherical Harmonic Transform for 3D rotations Energy Energy Frequency Frequency Circular Power Spectrum Spherical Power Spectrum

10. Translation Invariance 1D Function

11. Translation Invariance … = + + + + 1D Function Cosine/Sine Decomposition

12. Translation Invariance … = + + + + 1D Function = Constant Frequency Decomposition

13. Translation Invariance … = + + + + + 1D Function + = Constant 1st Order Frequency Decomposition

14. Translation Invariance … = + + + + + 1D Function + + = Constant 1st Order 2nd Order Frequency Decomposition

15. Translation Invariance … = + + + + + 1D Function … + + + + = Constant 1st Order 2nd Order 3rd Order Frequency Decomposition

16. Translation Invariance Amplitudes invariantto translation … = + + + + + 1D Function … + + + + = Constant 1st Order 2nd Order 3rd Order Frequency Decomposition

17. = + + + 3rd Order Constant 1st Order 2nd Order + + + Rotation Invariance • Represent each spherical function as a sum of harmonic frequencies (orders)

18. Rotation Invariance • Store “how much” (L2-norm) of the shape resides in each frequency to get a rotation invariant representation 3rd Order Constant 1st Order 2nd Order = + + +

19. Power Spectrum • Translation-invariance: • Represent the model in a Cartesian coordinate system • Compute the 3D Fourier transform • Store the amplitudes of the frequency components Cartesian Coordinates y Translation Invariant Representation z x

20. Power Spectrum • Single axis rotation-invariance: • Represent the model in a cylindrical coordinate system • Compute the Fourier transform in the angular direction • Store the amplitudes of the frequency components q Cylindrical Coordinates r h Rotation Invariant Representation

21. Power Spectrum • Full rotation-invariance: • Represent the model in a spherical coordinate system • Compute the spherical harmonic transform • Store the amplitudes of the frequency components q Spherical Coordinates f r Rotation Invariant Representation

22. Power Spectrum • Power spectrum representations • Are invariant to transformations • Give a lower bound for the best match • Tend to discard too much information • Translation invariant: n3 data -> n3/2 data • Single-axis rotation invariant: n3 data -> n3/2 data • Full rotation invariant: n3 data -> n2 data

23. Shells Histogram Power Spectrum

24. Outline • Matching • Alignment • Exhaustive Search • Invariance • Normalization • Part vs. Whole • Conclusion

25. Translation Scale Rotation Normalization • Place a model into a canonical coordinate frame by normalizing for: • Translation • Scale • Rotation

26. Normalization [Horn et al., 1988] • Place a model into a canonical coordinate frame by normalizing for: • Translation: Center of mass • Scale • Rotation Initial Models Translation-Aligned Models

27. Normalization [Horn et al., 1988] • Place a model into a canonical coordinate frame by normalizing for: • Translation • Scale: Mean variance • Rotation Translation-Aligned Models Translation- and Scale-Aligned Models

28. Normalization • Place a model into a canonical coordinate frame by normalizing for: • Translation • Scale • Rotation: PCA alignment PCA Alignment Translation- and Scale-Aligned Models Fully Aligned Models

29. Rotation • Properties: • Translation and rotation normalization is guaranteed to give the best alignment • Rotation normalization is ambiguous PCA Alignment Directions of the axes are ambiguous

30. Normalization (PCA) • PCA defines a coordinate frame up to reflection in the coordinate axes. • Make descriptor invariant to axial-reflections • Reflections fix the cosine term • Reflections multiply the sine term by -1 y Translation Invariant Representation z x

31. Retrieval Results (Rotation) Size: Gaussian EDT Precision Time: Recall

32. Alignment • Exhaustive search: • Best results • Inefficient to match • Normalization: • Provably optimal for translation and scale • Works well for rotation if models have well defined principal axes and the directional ambiguity is resolved • Invariance: • Compact • Efficient • Often less discriminating

33. Outline • Matching • Alignment • Exhaustive Search • Invariance • Normalization • Part vs. Whole • Conclusion

34. Partial Shape Matching • Cannot use global normalization methods that depend on whole model information: • Center of mass for translation • Mean variance for scale • Principal axes for rotation Normalized Whole Normalized Part (Mis-)Aligned Models

35. Partial Shape Matching • Cannot use global normalization methods that depend on whole model information: • Exhaustively search for best alignment • Normalize using local shape information • Use transformation invariant representations Normalized Whole Normalized Part (Mis-)Aligned Models

36. Spin Images & Shape Contexts • Translation (Exhaustive Search): • Represent each database model by many descriptors centered at different points on the surface. Model Multi-Centered Descriptors

37. Spin Images & Shape Contexts • Translation (Exhaustive Search): • To match, center at a random point on the query and compare against the different descriptors of the target Randomly-Centered Descriptor Query Part Best Match Target Descriptor

38. Spin Images & Shape Contexts • Rotation (Normalization): • For each center, represent in cylindrical coordinates about the normal vector • [Spin Images]: Store energy in each ring • [Harmonic Shape Contexts]: Store power spectrum of each ring • [3D Shape Contexts]: Search over all rotations about the normal for best match n n n

39. Spin Images & Shape Contexts Image courtesy of Frome et al, 2003 • Spin images and shape contexts allow for part-in-whole searches by exhaustively searching for translation and using the normal for rotation alignment • [Spin Images]: Store energy in each ring • [Harmonic Shape Contexts]: Store power spectrum of each ring • [3D Shape Contexts]: Search over all rotations about the normal for best match

40. Conclusion • Aligning Models: • Exhaustive Search • Normalization • Invariance • Partial Object Matching • Can’t use global normalization techniques • Translation: Exhaustive Search • Rotation: Normal + Exhaustive/Invariant

41. Conclusion • Shape Descriptors and Model Matching: • Decoupling representation from registration • Can design and evaluate descriptors without having to solve the alignment problem • Can develop methods for alignment without considering specific shape descriptors