Camera Calibration

1. Camera Calibration March 6, 2007

2. Camera Calibaration • Finding camera’s internal parameters that effect the imaging process. • Position of the image center on the image (usually not just (width/2, height/2). • Focal length • Scaling factors for row and column pixels. • Lens distortion.

3. Row and Column Pixels Scaling • Camera pixels are not necessarily squares • Analog output and digitalizing.

4. Camera Calibaration Each camera can be considered as a function: a function that takes each 3D point to a point in 2D image plane. (x, y, z) ------ > (X, Y) Camera calibration is about finding (or approximating) this function. Difficulty of the problem depends on the assumed form of this function, e.g., perspective model, radial distortion of the lense.

5. Camera Calibaration • Good camera calibration is needed when we want to reconstruct the geometry from images • Robotics, human-robot interaction • Robotics, robot navigation. • Location of objects in 3D

6. Example: Locating feature points in 3D Suppose the camera is calibrated and the 3D positions of the feature points are known for the first frame

7. Example: Locating feature points in 3D

8. Camera Calibration • Literature on this subject is tremendous. Hundreds of published papers. • Linear algebraic methods present here • Can be used as initialization for iterative non-linear method. • Vanishing points and projective geometry

9. Calibration Procedure Calibration: finding the function (defined by the camera) that maps 3D points to 2D image plane. First thing required is to obtain pairs of corresponding 3D and 2D points. (X1, Y1, Z1) (x1, y1), (X2, Y2, Z2) (x2, y2), ….

10. Calibration Procedure • Calibration Target: Two perpendicular planes with chessboard pattern. • We know the 3D positions of the corners with respect to a coordinates system defined on the target. • Place a camera in front of the target and we can locate the corresponding corners on the image. This gives us the correspondences. • Recover the equation that describes imaging projection and camera’s internal parameters. At the same time, also recover the relative orientation between the camera and the target (pose).

11. Finding Corners • Corner detector • Canny Edge detector plus fitting lines to the detected edges. Find the intersections of the lines. • Manual input. Matching 3D and 2D points (we know the number of corners) by counting. This gives corresponding pairs ( world point) < --- > (image point ) (X1, Y1, Z1) (x1, y1),

12. World Coordinates and Camera Coordinates

13. Camera Frame to World Frame Rotation matrix

14. Homogeneous Coordinates Let C = - Rt T

15. Perspective Projection Use Homogeneous coordinates, the perspective projection becomes linear.

16. Pixel Coordinates

17. Calibration Matrix

18. Putting Everything Together

19. Calibration • Estimate matrix P using scene points and their images • Estimate the intrinsic and extrinsic parameters • Left 3x3 submatrix is the product of an upper triangular matrix and an orthogonal matrix.

20. Computing the Matrix P

21. Computing the Matrix P

22. Computing the Matrix P

23. Computing the Matrix P

24. Computing the translation component

25. Computing the Matrix P We know P and we can get the translation C. Then we get M = KR. Find the QR decomposition of M to get K and R !! Further Improvement

26. Vanishing Points and Image Center Parallel lines in 3D has a vanishing point on the 2D image plane. Point of intersections of these lines. Take three bundles of mutually perpendicular lines in 3D and compute the three vanishing points. The image center is the orthocenter of the triangle formed by the 3 vanishing points!! Homework 3