1. Static Image Mosaicing Amin Charaniya (amin@cse.ucsc.edu) EE 264: Image Processing and Reconstruction

2. Presentation Overview • Problem definition • Background • Literature Survey • Image transformations • Image Registration • Coarse Image registration • Transformation Optimization • Image Blending • Implementation and Results • Conclusions (limitations and enhancements)

3. The Problem Q: “Static” ? Ans.: No moving objects in the scene. Image 1 Image 2 + Mosaiced image

4. Original images Image Registration / Alignment / Warping Image Blending The Solution

5. Constraints • Scene • Static / Dynamic • Planar / Non planar (perspective distortion) • Camera Motion • Translation (sideways motion) • Panning and Tilting (rotation about the Y and X axes) • Scaling (zooming, forward / backward motion) • General motion • Other Constraints • Automated / User input

6. Background and Literature survey • Barnea & Silverman, 1972 (L1 Norm) • Kuglin & Hines, 1975 (Phase Correlation) • Mann & Picard, 1994 (Cylindrical projection) • Irani & Anandan, 1995 (Static and Dynamic mosaics) • Szeliski, 1996 (Transformation optimization) • Badra, 1998 (Rotation and Zooming) • Peleg and Rousso, 2000 (Adaptive Manifolds, Mosaicing using strips)

7. Input image Output image Transformation Original shape Rigid transformation Affine transformation Projective transformation Image transformations

8. Presentation Overview • Problem definition • Background • Literature Survey • Image transformations • Image Registration • Coarse Image registration • Transformation Optimization • Image Blending • Implementation and Results • Conclusions (limitations and enhancements)

9. Transformation Optimization Initial transformation { Error Improved ? Phase Correlation L1 Norm User input Image Registration Coarse Image Registration

10. d(x,y) Inverse transform maximum (x0, y0) Phase Correlation • Kuglin & Hines, 1975 • Translation property of Fourier Transform

11. Spatial Correlation, L1 Norm • Barnea and Silverman f2 f2 E(x0,y0) = |f1(x,y) – f2(x- x0, y- y0)| f1 • Spatial correlation techniques • User input

12. minimize Compute partial derivatives Transformation Optimization • Richard Szeliski, “Video Mosaics for Virtual Environments”, 1996. • Optimization of initial transformation matrix M, to minimize error. • Levenberg-Marquardt non-linear minimization algorithm.

13. Transformation Optimization • Advantages • Faster convergence • Statistically optimal solution • Limitations • Local minimization (need a good initial guess)

14. Presentation Overview • Problem definition • Background • Literature Survey • Image transformations • Image Registration • Coarse Image registration • Transformation Optimization • Image Blending • Implementation and Results • Conclusions (limitations and enhancements)

15. Image Blending • Smooth transition (edges, illumination artifacts) • Simple averaging • Weighted averaging • Sample weight function – “hat filter” 0 xmax • More weight at the center of the image, less at the edges

16. Image blending Simple averaging Weighted averaging

17. Presentation Overview • Problem definition • Background • Literature Survey • Image transformations • Image Registration • Coarse Image registration • Transformation Optimization • Image Blending • Implementation and Results • Conclusions (limitations and enhancements)

18. Implementation • Implemented using Matlab • Source Images • BE 230 lab images (fixed tripod) • College 8 images (free hand motion, perpective distortion) • East Field House images (free hand motion) • Equipment: Sony DCR-TRV 900 3CCD digital camcorder

19. Sample results

20. Sample results

21. Conclusions/Enhancements • Better automatic coarse registration techniques needed. • Need to handle more general camera motion.

22. Thanks for listening !! Questions ?