Optical Flow

# Optical Flow

## Optical Flow

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
##### Presentation Transcript

1. Optical Flow Marc Pollefeys COMP 256 Some slides and illustrations from L. Van Gool, T. Darell, B. Horn, Y. Weiss, P. Anandan, M. Black, K. Toyama

2. last week: polar rectification

3. Last week: polar rectification

4. Similarity measure (SSD or NCC) Optimal path (dynamic programming ) Last week: Stereo matching • Constraints • epipolar • ordering • uniqueness • disparity limit • disparity gradient limit • Trade-off • Matching cost (data) • Discontinuities (prior) (Cox et al. CVGIP’96; Koch’96; Falkenhagen´97; Van Meerbergen,Vergauwen,Pollefeys,VanGool IJCV‘02)

5. Questions for assignment?

6. Tentative class schedule

7. Optical Flow • Brightness Constancy • The Aperture problem • Regularization • Lucas-Kanade • Coarse-to-fine • Parametric motion models • Direct depth • SSD tracking • Robust flow • Bayesian flow

8. Optical Flow:Where do pixels move to?

9. Motion is a basic cue Motion can be the only cue for segmentation

10. Motion is a basic cue Even impoverished motion data can elicit a strong percept

11. Applications • tracking • structure from motion • motion segmentation • stabilization • compression • mosaicing • …

12. Optical Flow • Brightness Constancy • The Aperture problem • Regularization • Lucas-Kanade • Coarse-to-fine • Parametric motion models • Direct depth • SSD tracking • Robust flow • Bayesian flow

13. Definition of optical flow OPTICAL FLOW = apparent motion of brightness patterns Ideally, the optical flow is the projection of the three-dimensional velocity vectors on the image 

14. Caution required ! Two examples : 1. Uniform, rotating sphere  O.F. = 0 2. No motion, but changing lighting  O.F.  0 

15. Caution required !

16. Mathematical formulation I (x,y,t) = brightness at (x,y) at time t Brightness constancy assumption: Optical flow constraint equation : 

17. Optical Flow • Brightness Constancy • The Aperture problem • Regularization • Lucas-Kanade • Coarse-to-fine • Parametric motion models • Direct depth • SSD tracking • Robust flow • Bayesian flow

18. The aperture problem 1 equation in 2 unknowns 

19. The aperture problem

20. Remarks

21. What is Optic Flow, anyway? • Estimate of observed projected motion field • Not always well defined! • Compare: • Motion Field (or Scene Flow) projection of 3-D motion field • Normal Flow observed tangent motion • Optic Flow apparent motion of the brightness pattern (hopefully equal to motion field) • Consider Barber pole illusion

22. Optical Flow • Brightness Constancy • The Aperture problem • Regularization • Lucas-Kanade • Coarse-to-fine • Parametric motion models • Direct depth • SSD tracking • Robust flow • Bayesian flow

23. Horn & Schunck algorithm Additional smoothness constraint : besides OF constraint equation term minimize es+ec 

24. so the Euler-Lagrange equations are is the Laplacian operator Horn & Schunck The Euler-Lagrange equations : In our case , 

25. Horn & Schunck Remarks : 1. Coupled PDEs solved using iterative methods and finite differences 2. More than two frames allow a better estimation of It 3. Information spreads from corner-type patterns 

26. Horn & Schunck, remarks 1. Errors at boundaries 2. Example of regularisation (selection principle for the solution of illposed problems) 

27. Results of an enhanced system

28. Structure from motion with OF

29. Optical Flow • Brightness Constancy • The Aperture problem • Regularization • Lucas-Kanade • Coarse-to-fine • Parametric motion models • Direct depth • SSD tracking • Robust flow • Bayesian flow

30. Optical Flow • Brightness Constancy • The Aperture problem • Regularization • Lucas-Kanade • Coarse-to-fine • Parametric motion models • Direct depth • SSD tracking • Robust flow • Bayesian flow

31. Optical Flow • Brightness Constancy • The Aperture problem • Regularization • Lucas-Kanade • Coarse-to-fine • Parametric motion models • Direct depth • SSD tracking • Robust flow • Bayesian flow