1 / 74

Image Stitching

Image Stitching. Panoramic Image Mosaics. Full screen panoramas (cubic): http://www.panoramas.dk/ Mars: http://www.panoramas.dk/fullscreen3/f2_mars97.html 2003 New Years Eve: http://www.panoramas.dk/fullscreen3/f1.html. Gigapixel panoramas & images. Mapping / Tourism / WWT.

ocean
Télécharger la présentation

Image Stitching

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Image Stitching

  2. Panoramic Image Mosaics Full screen panoramas (cubic): http://www.panoramas.dk/ Mars: http://www.panoramas.dk/fullscreen3/f2_mars97.html 2003 New Years Eve: http://www.panoramas.dk/fullscreen3/f1.html Image Stitching

  3. Gigapixel panoramas & images Mapping / Tourism / WWT Medical Imaging Image Stitching

  4. Image Mosaics • + + … + = Goal: Stitch together several images into a seamless composite Image Stitching

  5. Today’s lecture • Image alignment and stitching • motion models • image warping • point-based alignment • complete mosaics (global alignment) • compositing and blending • ghost and parallax removal Image Stitching

  6. Readings • Szeliski, CVAA: • Chapter 3.6: Image warping • Chapter 5.1: Feature-based alignment (in preparation) • Chapter 9.1: Motion models • Chapter 9.2: Global alignment • Chapter 9.3: Compositing • Recognizing Panoramas, Brown & Lowe, ICCV’2003 • Szeliski & Shum, SIGGRAPH'97 Image Stitching

  7. Motion models

  8. Motion models • What happens when we take two images with a camera and try to align them? • translation? • rotation? • scale? • affine? • perspective? • VideoMosaic Image Stitching

  9. Image Warping

  10. f f f h h x x x f x Image Warping • image filtering: change range of image • g(x) = h(f(x)) • image warping: change domain of image • g(x) = f(h(x)) Image Stitching

  11. h h Image Warping • image filtering: change range of image • g(x) = h(f(x)) • image warping: change domain of image • g(x) = f(h(x)) f g f g Image Stitching

  12. Parametric (global) warping • Examples of parametric warps: aspect rotation translation perspective cylindrical affine Image Stitching

  13. 2D coordinate transformations Image Stitching

  14. 2D coordinate transformations • Translation • Rotation + translation • Similarity • Affine Image Stitching

  15. p’ H p Homography • A: Projective – mapping between any two PPs with the same center of projection • rectangle should map to arbitrary quadrilateral • parallel lines aren’t • but must preserve straight lines • same as: project, rotate, reproject • called Homography PP2 PP1 • To apply a homography H • Compute p’ = Hp (regular matrix multiply) • Convert p’ from homogeneous to image coordinates

  16. 2D coordinate transformations • Projective transformation or Homography • Normalized homogeneous Homogeneous, only defined up to a scale Image Stitching

  17. 2D coordinate transformations Image Stitching

  18. Image Warping • Given a coordinate transform x’ = h(x) and a source image f(x), how do we compute a transformed image g(x’)=f(h(x))? h(x) x x’ f(x) g(x’) Image Stitching

  19. Forward Warping • Send each pixel f(x) to its corresponding location x’=h(x) in g(x’) • What if pixel lands “between” two pixels? h(x) x x’ f(x) g(x’) Image Stitching

  20. Forward Warping • Send each pixel f(x) to its corresponding location x’=h(x) in g(x’) • What if pixel lands “between” two pixels? h(x) x x’ f(x) g(x’) Image Stitching

  21. Forward Warping • Non-integer x’ • Nearest integer coordinate • Aliasing • Jumping • Splatting • Distribute to 4 neighbors (weighted) and normalize later • Both aliasing and blur • Cracks and holes • Filling hole with nearby neighbors • Further aliasing and blurring

  22. Inverse Warping • Get each pixel g(x’) from its corresponding location x’=h(x) in f(x) • What if pixel comes from “between” two pixels? h(x) x x’ f(x) g(x’) Image Stitching

  23. Inverse Warping • Get each pixel g(x’) from its corresponding location x’=h(x) in f(x) • What if pixel comes from “between” two pixels? • Answer: resample color value from interpolated (prefiltered) source image x x’ f(x) g(x’) Image Stitching

  24. Interpolation • Possible interpolation filters: • nearest neighbor • bilinear • bicubic (interpolating) • sinc / FIR • Needed to prevent “jaggies” and “texture crawl” Image Stitching

  25. Interpolation Bicubic • Bilinear ɑ: the derivative at x=1 Image Stitching

  26. Interpolation • Window sinc functions Image Stitching

  27. Image Interpolation All perform high-accuracy low-pass filtering while preserve details and avoid aliasing Bilinear for speed Bicubic and windowed sinc for better visual quality Image Stitching

  28. Decimation • Interpolation: increase the image resolution • Decimation (downsampling): reduce resolution Samples are convolved with a low-pass filter before downsampled to avoid aliasing Image Stitching

  29. Decimation • Sample rate : same kernel as interpolation Commonly use r=2 Image Stitching

  30. Decimation Frequency response Image Stitching

  31. Decimation • Linear filter gives a relatively poor response • Binomial filter cuts off a lot of frequencies but is useful for computer vision analysis pyramid • Cubic filter: a=-1 filter has a shaper fall-off than the a=-0.5 filter • Cosine-windowed sinc function • QMF-9 filter for wavelet denoising and aliases a fair amount • 9/7 analysis filter form JPEG2000 Image Stitching

  32. Prefiltering • Essential for downsampling (decimation) to prevent aliasing • MIP-mapping [Williams’83]: • build pyramid (but what decimation filter?): • block averaging • Burt & Adelson (5-tap binomial) • 7-tap wavelet-based filter (better) • trilinear interpolation • bilinear within each 2 adjacent levels • linear blend between levels (determined by pixel size) Image Stitching

  33. Prefiltering • Essential for downsampling (decimation) to prevent aliasing • Other possibilities: • summed area tables • elliptically weighted Gaussians (EWA) [Heckbert’86] Image Stitching

  34. Resmaple on interpolated signal • To avoid blurring or aliasing Image Stitching

  35. Motion models (reprise)

  36. Affine Perspective 3D rotation Translation 2 unknowns 6 unknowns 8 unknowns 3 unknowns Motion models Image Stitching

  37. Planar perspective motion • Simplest case: • Overlapping photographic prints, scanned pieces • Translation and rotation are usually adequate Four unknowns Image Stitching

  38. Planar perspective motion • Planar Homography Image Stitching

  39. Planar Perspective Motion • Extract features from images • Match features using RANSAC and get inliers • Estimate a homography H given pairs of corresponding features • Least squares fitting Image Stitching

  40. Solving for homography p’ = Hp Can set scale factor i=1. So, there are 8 unknowns. Set up a system of linear equations: Ah = b where vector of unknowns h = [a,b,c,d,e,f,g,h]T Need at least 8 eqs, but the more the better… Solve for h. If overconstrained, solve using least-squares Image Stitching

  41. Image warping with homographies image plane in front image plane below Image Stitching

  42. Plane perspective mosaics • 8-parameter generalization of affine motion • works for pure rotation or planar surfaces • Limitations: • local minima • slow convergence • difficult to control interactively Image Stitching

  43. Rotational mosaics • Directly optimize rotation and focal length • Advantages: • ability to build full-view panoramas • easier to control interactively • more stable and accurate estimates GrandCanyon panorama Image Stitching

  44. (Xc,Yc,Zc) uc f u 3D → 2D Perspective Projection Image Stitching

  45. Rotational panoramas • Setting With known f0, f1, solve R10 Image Stitching

  46. Rotational panoramas • More stable for large-scale image stitching algorithms than homography • How to solve the unknowns? • Update R10 with incremental rotation matrix R(w) to current estimate R10 Image Stitching

  47. Rotations and quaternions • How do we represent rotation matrices? • Axis / angle (n,θ)R = I + sinθ [n] + (1- cosθ) [n]2(Rodriguez Formula), with[n] = cross product matrix (see paper) • Unit quaternions [Shoemake SIGG’85]q = (nsinθ/2, cosθ/2) = (w,s)quaternion multiplication (division is easy)q0 q1 = (s1w0+ s0w1, s0 s1-w0·w1) Image Stitching

  48. Incremental rotation update • Small angle approximationΔR = I + sinθ [n] + (1- cosθ) [n]2 ≈ θ [n] = [ω]linear inω • Update original R matrixR ← RΔR Image Stitching

  49. Rotational panoramas • How to solve the unknowns of R10? • Compute the incremental rotation vector w, Image Stitching

  50. Rotational mosaic • Projection equations • Project from image to 3D ray • (x0,y0,z0) = (u0-uc,v0-vc,f) • Rotate the ray by camera motion • (x1,y1,z1) = R01 (x0,y0,z0) • Project back into new (source) image • (u1,v1) = (fx1/z1+uc,fy1/z1+vc) Image Stitching

More Related