Midterm Presentation

1. Midterm Presentation Vitaliy Orekhov Imaging, Robotics, & Intelligent Systems Laboratory The University of Tennessee October 7, 2005

2. Tasks • ECE503 – Modern Transforms • ECE572 – Digital Image Processing • ECE573 –Test, Evaluate, and Transfer Camera Calibration Code to C++ • ECE672 – Write Survey on Wide Angle Camera Calibration and Image Correction

3. Test, Evaluate, and Transfer Camera Calibration Code to C++ • Calibration code was originally put together by Chris Broaddus • Written in MATLAB • Main sections/algorithms • Corner Detection and subpixel refinement • Homography estimation • Solving for intrinsic parameters • Solving for extrinsic parameters • Solving for the lens projection estimation • Decentering distortion • Complete model • Bundle adjustment • Model selection f = focal length mx,y = camera pixel size Px,y = Principal point coordinates S = skew parameter

4. Test, Evaluate, and Transfer Camera Calibration Code to C++ • Calibration code was originally put together by Chris Broaddus • Written in MATLAB • Main sections/algorithms • Corner Detection and subpixel refinement • Homography estimation • Solving for intrinsic parameters • Solving for extrinsic parameters • Solving for the lens projection estimation • Decentering distortion • Complete model • Bundle adjustment • Levenberg Marquardt algorithm • Model selection R = rotation matrix D = coordinates of camera center in world coordinates

5. OpenCV Intel’s Open Source Computer Vision (OpenCV) Library • OpenCV is cross-platform middle-to-high level API that consists of a few hundred C functions • Put together so that efforts of the vision community can be consolidated and performance optimized • Open source library is mainly aimed at real-time computer vision

6. Status • Getting familiar with C++ and Microsoft Visual C++ • Looked into different Image Libraries and the functions already implemented in C++ • OPENCV, VXL, IMLAB, CImg and others • Testing MATLAB code Next steps: • Continue getting familiar with C++ and the image libraries • Transfer sections of calibration code to Visual C++

7. Task4Survey on Wide Angle Camera Calibration and Image Correction

8. Wide-Angle Lenses • Easier to map local information for visual search, navigation, or detection • Features of wide-angle lenses • Background objects seem far away and small • Front parts of objects appear unnaturally prominent • Straight lines near the edges appear curved Image: by Chris Broaddus Image: from Presidents and Fellows of Harvard

9. Omni-directional Vision • Desired for tracking/observing multiple moving targets or objects • Camera network • Panning a camera takes time and is not sufficient for real time applications • Single omni-directional camera (catadioptric)

10. Areas of Application • Navigation for Robots • Just like for insects and arthropods large FOV provides advantages over other approaches • Stereo Reconstruction • Stereo panoramas provide the ability to freely view in stereo, in all directions • Camera rotated about axis • Multiple camera system • Video surveillance • Monitoring dynamically changing environments • Operate in a network of cameras to provide wide scene coverage Image: “Automatic Disparity Control in Stereo Panoramas (OmniStereo)”, Yael Pritch Moshe Ben-Ezra, Shmuel Peleg Image: “Construction of an Immersice Mixed Environment Using an Omnidirectional Stereo Image Sensor” Jun Shimamura Harue Takemura

11. Calibration for Computer Vision • It is a necessary step in 3D computer vision to extract metric information from 2D images • Before the camera can be used for precise computer vision applications, the camera needs to be characterized • How does a point in 3D world coordinates get projected onto the camera imaging plane

12. Camera Calibration • Camera calibration is the process of calculating the intrinsic and extrinsic parameters of a camera • The intrinsic parameters describe the internal characteristics of the camera • focal length, camera pixel size, principal point coordinates, and skew parameter • The extrinsic parameters describe the position of the camera in the world • six parameters: three are for position of the center of projection, and three are for orientation of the image plane coordinate frame

13. Calibration Techniques • Photogrammetric Calibration • Performed by using a known calibration object • Uses two or three planes orthogonal to each other • Self-Calibration • Uses consecutive images of the environment as the reference to find intrinsic parameters • Uses few or none assumptions about particular structure of the scene • Known camera motions in a static scene • Both techniques can be combined • Use 2D metric information and unrestricted motion

14. Projective Camera Model

15. Radial Distortion In general, radial distortion is approximated by: Distortion function is mostly dominated by the first terms.

16. Lens Projection In general, the lens projection is approximated by:

17. Status • Calibration of Multiple Cameras • Architectures for Real Time Correction of Wide-Angle Camera Images • Improvement of the calibration accuracy for wide angle lenses • Integration of pan/tilt/zoom camera • Improve resolution and viewing angle • Robust and accurate calibration required

18. Thank you Suggestions/Comments/Questions