Camera Models

1. Camera Models CMPUT 498/613 Richard Hartley and Andrew Zisserman, Multiple View Geometry, Cambridge University Publishers, 2000 Readings: HZ Ch 6, 7

2. Hierarchy of cameras xorth xwp xparap Weak perspective: Orthographic: Para-perspective: First order approximation of perspective Object plane Image plane xpersp X0(origin) Camera center Perspective:

3. Examples of camera projections perspective Orthographic (parallel)

4. Animal eye: a looonnng time ago. Photographic camera: Niepce, 1816. Pinhole perspective projection: Brunelleschi, XVth Century. Camera obscura: XVIth Century.

5. Pinhole camera model

6. Pinhole camera model

7. Principal point offset principal point

8. Principal point offset calibration matrix

9. Camera rotation and translation

10. CCD camera

11. non-singular Finite projective camera 11 dof (5+3+3) decompose P in K,R,C? {finite cameras}={P4x3 | det M≠0} If rank P=3, but rank M<3, then cam at infinity

12. Camera anatomy Camera center Column points Principal plane Axis plane Principal point Principal ray

13. Column vectors Image points corresponding to X,Y,Z directions and origin

14. Row vectors note: p1,p2 dependent on image reparametrization Camera center null-space camera projection matrix, (3 plane intersection)

15. principal point The principal point

16. (pseudo-inverse) Action of projective camera on point Forward projection Back-projection

17. =( )-1= -1 -1 R R Q Q Camera matrix decomposition Finding the camera center (use SVD to find null-space) Finding the camera orientation and internal parameters (use RQ decomposition ~QR) (if only QR, invert)

18. When is skew non-zero? arctan(1/s) g 1 for CCD/CMOS, almost always s=0 Image from image, s≠0 possible (non coinciding principal axis) resulting camera:

19. Euclidean vs. projective general projective interpretation Meaningfull decomposition in K,R,t requires Euclidean image and space Camera center is still valid in projective space Principal plane requires affine image and space Principal ray requires affine image and Euclidean space

20. Cameras at infinity Camera center at infinity Affine and non-affine cameras Definition: affine camera has P3T=(0,0,0,1)

21. Affine cameras

22. Affine cameras modifying p34 corresponds to moving along principal ray

23. Affine cameras now adjust zoom to compensate

24. Error in employing affine cameras point on plane parallel with principal plane and through origin, then general points

25. Affine imaging conditions • Approximation should only cause small error • D much smaller than d0 • Points close to principal point • (i.e. small field of view)

26. Decomposition of P∞ absorb d0 in K2x2 alternatives, because 8dof (3+3+2), not more

27. Summary parallel projection canonical representation calibration matrix principal point is not defined

28. A hierarchy of affine cameras Orthographic projection (5dof) Scaled orthographic projection (6dof)

29. A hierarchy of affine cameras Weak perspective projection (useful infinite appr. of unknown CCD camera) (7dof)

30. A hierarchy of affine cameras Affine camera (8dof) • Affine camera=camera with principal plane coinciding with P∞ • Affine camera maps parallel lines to parallel lines • No center of projection, but direction of projection PAD=0 • (point on P∞)

31. Hierarchy of cameras xorth xwp xparap Weak perspective: Orthographic: Para-perspective: First order approximation of perspective Object plane Image plane xpersp X0(origin) Camera center Perspective:

32. Other Cameras Pushbroom cameras (11dof) Straight lines are not mapped to straight lines! (otherwise it would be a projective camera) Line cameras (5dof) Null-space PC=0 yields camera center Also decomposition

33. A 3D Vision Problem: Multi-view geometry - resection • Projection equation xi=PiX • Resection: • xi,X Pi Given image points and 3D points calculate camera projection matrix.

34. Estimating camera matrix P • Given a number of correspondences between 3-D points and their 2-D image projections Xi xi, we would like to determine the camera projection matrixP such that xi =PXifor all i

35. A Calibration Target Y xi Xi Z X courtesy of B. Wilburn

36. Estimating P: The Direct Linear Transformation (DLT) Algorithm • xi =PXiis an equation involving homogeneous vectors, so PXi and xi need only be in the same direction, not strictly equal • We can specify “same directionality” by using a cross product formulation:

37. DLT Camera Matrix Estimation: Preliminaries • Let the image point xi = (xi, yi, wi)T (remember that Xi has 4 elements) • Denoting the jth row of P by pjT (a 4-element row vector), we have:

38. DLT Camera Matrix Estimation: Step 1 • Then by the definition of the cross product, xi PXi is:

39. DLT Camera Matrix Estimation: Step 2 • The dot product commutes, so pjTXi=XTipj, and we can rewrite the preceding as:

40. DLT Camera Matrix Estimation: Step 3 • Collecting terms, this can be rewritten as a matrix product: where 0T = (0, 0, 0, 0). This is a 3 x 12 matrix times a 12-element column vector p = (p1T, p2T, p3T)T

41. What We Just Did

42. DLT Camera Matrix Estimation: Step 4 • There are only two linearly independent rows here • The third row is obtained by adding xi times the first row to yi times the second and scaling the sum by -1/wi

43. DLT Camera Matrix Estimation: Step 4 • So we can eliminate one row to obtain the following linear matrix equation for the ith pair of corresponding points: • Write this as Aip = 0

44. DLT Camera Matrix Estimation: Step 5 • Remember that there are 11 unknowns which generate the 3 x 4 homogeneous matrix P (represented in vector form by p) • Each point correspondence yields 2 equations (the two row of Ai) • We need at least 5 ½ point correspondences to solve for p • Stack Ai to get homogeneous linear system Ap = 0

45. Experiment: