1. EE 7700 Geometric Transformations

2. Geometric Transformation translation Rotation matrix scale Scale matrix rotation & scale Rigid flow

3. Affine Flow

4. Perspective Flow

8. EE 7730 RANSAC: RANdom SAmple Consensus

9. Outliers • Consider the least squares fit for optical flow: If some of the values are wrong, it will degrade the estimation.

10. Outliers • It is best not to include outliers in the estimation. Line Fitting Problem: Given (x1,y1), …, (xN,yN); find the line y=ax+b Outliers Best fit is degraded due to the outliers

11. Robust Estimation • Two step process: • Classify data points as outliers or inliers • Use inliers only to fit a model

12. RANSAC • Repeat for k times: • Randomly choose n points (the smallest number of points required) from the data. • Estimate the parameters using these points. • For each data point other than these n points: • Check if the data point is within a threshold, t, distance of current model; if it is, the data point is consistent with current model. • The total number of data points that are consistent is model’s support. • If the support is larger than a predetermined number, d, then there is a good fit. Re-estimate the parameters using these inliers. • Choose the best fit with the smallest fitting error.

13. RANSAC Two samples and their supports for line-fitting

14. Example • Find the perspective parameters from Hartley & Zisserman

15. Example • Apply corner detectors to both images from Hartley & Zisserman

16. Example • Find the best match within a search window. from Hartley & Zisserman

17. Example • Initial match results from Hartley & Zisserman 188 matched features in left image pointing to locations of corresponding right image features

18. Example • Inliers and outliers after RANSAC from Hartley & Zisserman 89 outliers 99 inliers

19. Panoramic Image Reconstruction Find features Match features Fit parametric model Application: Mosaic construction

20. EE7730 Stereo Vision

21. Pinhole Camera

22. Review: Perspective Projection

23. Stereo scene point p p’ image plane optical center p p’

24. Epipolar Line p’ Y2 X2 Z2 O2 Epipole Stereo Constraints M Image plane Y1 p O1 Z1 X1 Focal plane

25. P p p’ O’ O From Geometry to Algebra All vectors shown lie on the same plane.

26. P p p’ O’ O From Geometry to Algebra

27. Matrix form of cross product a=axi+ayj+azk a×b=|a||b|sin(η)u b=bxi+byj+bzk

28. The Essential Matrix Essential matrix

29. disparity Depth Z Elevation Zw A Simple Stereo System LEFT CAMERA RIGHT CAMERA baseline Right image: target Left image: reference Zw=0

30. Parallel Cameras P Z xl xr f pl pr Ol Or Disparity: T T is the stereo baseline

31. Stereo View Right View Left View Disparity

32. (xl, yl) Correlation Approach LEFT IMAGE • For Each point (xl, yl) in the left image, define a window centered at the point

33. Correlation Approach RIGHT IMAGE (xl, yl) • … search its corresponding point within a search region in the right image

34. Correlation Approach RIGHT IMAGE (xr, yr) dx (xl, yl) • … the disparity (dx, dy) is the displacement when the correlation is maximum

35. Stereo results • Data from University of Tsukuba Scene Ground truth (Seitz)

36. Results with window correlation Estimated depth of field Ground truth (Seitz)

37. Results with better method • A state of the art method • Boykov et al., Fast Approximate Energy Minimization via Graph Cuts, • International Conference on Computer Vision, September 1999. Ground truth (Seitz)

38. Applications First-down line courtesy of Sportvision

39. Applications Virtual advertising courtesy of Princeton Video Image