Advanced Computer Vision

1. Advanced Computer Vision Structure from Motion

2. Structure from Motion • Geometric structure-from-motion problem: using image matches to estimate: • The 3D positions of the corresponding scene points in some fixed coordinate (the scene structure) • The projection matrices associated with the camera observing them (the motion of the points relative to the cameras)

3. Two Types • Affine structure from motion (Chapter 12) • Projective structure from motion (Chapter 13)

4. Basic Assumption • The cameras’ positions and possibly their intrinsic parameters are a priori unknown and may change over time. • Different from,say, stereopsis, where the camera parameters are known so we can focus on the correspondence problem. • Chapters 12 and 13 ignore the correspondence problem, assuming that the projections of n points have been matched across m pictures.

5. Applications • Image-based rendering: a video clip recorded by a hand-held camcorder, possibly zooming during the shoot, is used to capture the shape of an object and render it under new viewing conditions. (Chapter 26). • Active vision system: robot probes.

6. Problem: estimate the m 2x4 matrices Mi and the n positions Pj from the mxn correspondences pij . i j ij Affine Structure from Motion 2mn equations in 8m+3n unknowns Overconstrained problem, that can be solved using (non-linear) least squares!

7. If M and P are solutions, i j Ambiguity of Solution So are M’ and P’ where i j and Q is an affine transformation.

8. Problem: estimate the m 3x4 matrices M and the n positions P from the mn correspondences p . i j ij The Projective Structure-from-Motion Problem Given m perspective images of n fixed points P we can write j 2mn equations in 11m+3n unknowns Overconstrained problem, that can be solved using (non-linear) least squares!