Midterm Presentation

1. Midterm Presentation Vitaliy Orekhov Imaging, Robotics, & Intelligent Systems Laboratory The University of Tennessee March 07, 2006

2. Outline • Task 1 • Math472 – Modern Transforms • Task 2 • ECE571 – Pattern Classifications • Task 3 • ECE574 –Test, Evaluate, and Transfer Camera Calibration Code to C++ • Task 4 • ECE671 –Survey on Wide Angle Camera Calibration and Image Correction

3. www.oksolar.com www.fujinon.de Correcting Distortion • Correcting Distortion on an image sequence in real-time • IQeye3 Network Camera • 1.8M pixels/sec JPEG • 1/2" 1288 x 968 progressive scan CMOS digital imager • 50 fps (frames per second) at 160 by 120 • 19 fps at 320 by 240 • 5.4 fps at 640 by 480 • 3.6 fps at 800 by 600 • 1.3 fps at 1288 by 968 • YV2.2 x 1.4A-SA2 Lens • Fish-eye vari-focal lens with the horizontal field angle of 185-94 degrees (when used on 1/3 cameras)

4. Image Sequence Correction Diagram

5. Camera Calibration • Capturing sequence of images with calibration pattern • Extracting coordinates using Harris corner detections • Use MATLAB calibration code written by Chris Broaddus • Extract camera matrix • Distortion parameters • Radial and tangential distortion • Single polynomial approximation used to model the lens projections.

6. Image size: 620x620 185 deg. fov Image size: 620x620 80 deg. fov Image Correction • Code written in C++ • Creates a look up table based the camera matrix and distortion coefficients. • Code will work with most commonly used distortion models: • Look up table is used to correct each image captured from the camera in real-time

7. Wide Angle Camera Calibration • Image size 312x312 • Frame rate from camera: 14 fps. • Frame rate with distortion correction: 6 fps. • Smallest angle-of-view setting (about 80 deg.)

8. Wide Angle Camera Calibration • Image size 312x312 • Frame rate from camera: 14 fps. • Frame rate with distortion correction: 6 fps. • Widest angle-of-view setting (185deg.)

9. Wide Angle Camera Calibration • Original Image size 312x312 • Restored Image size 428x428 • Frame rate: 14 fps. • Distortion corrected not in real time • Smallest angle-of-view setting (about 80 deg.)

10. Processing Time Comparison 1* D. Eadie, F. Shevlin, A, Nisbet “Correction of Geometric Image Distortion Using FPGA’s” SPIE Proceeding Opto-Ireland, Vol. 4877,pp. 28-37, March 2003.

11. Task 4 Survey on Wide Angle Camera Calibration and Image Correction Fall 2005 Calibrating distortion in camera’s with wide field of view. Spring 2006 Complete calibration methods. • Test-range calibration • Non-metric calibration • Self calibration

12. Test-range Calibration • Methods which use calibration objects whose 2D or 3D coordinates are accurately known. Z. Zhang, “A flexible new Technique for camera calibration”, Technical Report MSR-TR-98-71, Microsoft Research, Dec. 1998. Z. Zhang, “A flexible new Technique for camera calibration”, IEEE Tans. Pattern Analysis and Machine Intelligence, Vol. 22, No. 11, 1330C1334, Nov. 2000. • Planar calibration pattern from at least two different views. • Two major steps • Closed form solution • Nonlinear optimization C. Broaddus, “Universal Geometrci Camera Calibration with Statistical Model Selection,” Masters Thesis, Department of Electrical Engineering, The Unverstity of Tennessee, Knoxville, TN 2005.

13. Non-metric and Self Calibration Non-metric calibration • These methods use invariants of image features. Self calibration • Methods which use camera intrinsic constraints, camera motion constraints or scene constraints. • Self calibration methods which use pure rotation have been shown to be very sensitive to radial distortion. B. Tordoff and D. Murray, “The impact of radial distortion on the self-calibration of rotating cameras,” CVIU, Vol. 96, No. 1, Oct. 2004, pp. 17-34.

14. Summary

15. Thank you Suggestions/Comments/Questions