Probabilistic Framework for Feature Point Matching and Its Applications in Tracking and Motion Analysis

1. Probabilistic Framework for Feature-point Matching Ron Tal, Minas E. Spetsakis York University

2. Outline • Introduction • Probabilistic Framework • Theoretical Validation • Our Algorithm • Results • Discussion

3. Introduction: The Problem • To determine inter-frame correspondence of points of interest • Applications include: • Tracking • Stereo • Motion Segmentation • Structure from Motion • Etc.

4. Introduction: Previous Approaches • Modeling optic flow as a set of linear constraints under simplified assumptions like: • Brightness constancy • Small motion • Smoothness • Lucas & Kanade, Horn & Schunk, etc. • Expanding the linear model in order to relax assumptions • Negahdaripour, GDIM, etc.

5. Introduction: Previous Approaches (cont.) • Modeling violations to the assumptions as ‘outliers’ and applying a framework of robust M-estimation • Black & Anandan • Handling larger motion using hierarchical application • Bergen et al., etc. • Enhanced parametric model, along with estimation • Jepson & Black using the EM to model multiple motions • Wills et al. use RANSAC, etc.

6. Probabilistic Framework: Rationale • A probabilistic model that associates feature appearance to its location in the image • Compute the probability of its location given its appearance and previous image(s) • Maximize this probability to get our answer • Parameters of the probabilistic model can be “learned”

7. Probabilistic Framework: In a Nutshell • Changes to feature appearance across frames can be modeled as a stochastic process

8. Probabilistic Framework: In a Nutshell • We can thus define the pdf that relates feature position given its appearance: • Where: • - Image appearance • - Statistical model of change in appearance • - Feature position • - Pixel values around the feature-point

9. Probabilistic Framework: In a Nutshell • Using Bayes’ Law, it becomes: • The denominator is constant with respect to position in the second frame • We thus determine the position by maximizing its joint probability with image appearance

10. Theoretical Validation: Recall the Classics • A very popular early solution to the problem has been given by Lucas & Kanade: • Assumptions include: • Small isotropicly distributed motion • Brightness constancy • Image noise is i.i.d

11. Theoretical Validation: Our Framework • Assuming uncertainty is isotropic and normal: becomes: • The covariance matrix can be partitioned:

12. Theoretical Validation: Our Framework • By performing inversion by partitioning and minimizing: • Applying Taylor approximation and the Woodbury identity:

13. Theoretical Validation: Our Framework • Substituting: • Since:

14. Theoretical Validation: Our Framework • We get:

15. Our Algorithm: Increasing Expressiveness • We want to relax as many assumptions as possible • We need to be able to model more complex changes to image appearance • A Gaussian Mixture Model can be used to approximate any pdf:

16. Our Algorithm: Increasing Expressiveness • Using a statistical noise model, we generate random samples that correspond to changes in image appearance due motion and sources of uncertainty • Using the EM Clustering algorithm, we approximate a GMM: 2 Components 3 Components

17. Our Algorithm: Noise Model • Random observations are synthesized using a noise model • Changes to image appearance modeled include, but not limited to: • The motion we want to determine • Sub-pixel jitter • Electronic camera noise • Brightness fluctuations

18. Results: Quantitative Evaluation • We select feature-points from the Middlebury Sequence exhibiting small to medium motion (5 pix ) • Additionally, we introduce i.i.d camera noise as well as global change of illumination • The performance of our method is compared with Hierarchical Horn & Schunk (red) and Black and Anandan (green)

19. Results: Quantitative Evaluation

20. Results: Qualitative Evaluation • Feature-point correspondence over multiple frames from handheld captured highway data • Example, point-tracking over 10 frames:

21. Results: Qualitative Evaluation • Result: 5 pix drift, overall motion is 38 pix • Single frame motion varies from 0.1 pix to 13 pix

22. Discussion: Performance • Framework shows great promise • Performs well under relaxed statistical assumptions • Algorithm can be easily customized for specific applications

23. Discussion: Performance Trade-offs • Increased accuracy of algorithm requires more generated samples and more components for the GMM • The cost is longer computational time for each iteration of EM • Correspondence of a single feature-point is insufficient for robust object tracking

24. Discussion: Future Work • Modification of GMM estimation to incorporate additional information and reduce the number of observations needed • Re-use pdf computed in previous frames to reduce computational cost • A tracker based on clustering of multiple feature-points using an affine motion model • Migration from MATLAB to OpenCV

25. Questions?