1 / 73

Lecture 02

Lecture 02. 08/11/2011 Sh ai Avidan. הבהרה: החומר המחייב הוא החומר הנלמד בכיתה ולא זה המופיע / לא מופיע במצגת. . Today. Alignment & warping 2d transformations Forward and inverse image warping Constructing mosaics Homographies Robust fitting with RANSAC. Main questions.

alicia
Télécharger la présentation

Lecture 02

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lecture 02 08/11/2011 ShaiAvidan הבהרה: החומר המחייב הוא החומר הנלמד בכיתה ולא זה המופיע / לא מופיע במצגת.

  2. Today • Alignment & warping • 2d transformations • Forward and inverse image warping • Constructing mosaics • Homographies • Robust fitting with RANSAC

  3. Main questions Warping: Given a source image and a transformation, what does the transformed output look like? T Alignment: Given two images with corresponding features, what is the transformation between them? T

  4. Motivation: Mosaics • Getting the whole picture • Consumer camera: 50˚ x 35˚ Slide from Brown & Lowe 2003

  5. Motivation: Mosaics • Getting the whole picture • Consumer camera: 50˚ x 35˚ • Human Vision: 176˚ x 135˚ Slide from Brown & Lowe 2003

  6. Motivation: Mosaics • Getting the whole picture • Consumer camera: 50˚ x 35˚ • Human Vision: 176˚ x 135˚ • Panoramic Mosaic = up to 360 x 180° Slide from Brown & Lowe 2003

  7. Warping problem • Given a set of points and a transformation, generate the warped image T(x,y) y y’ x x’ f(x,y) g(x’,y’) Figure Alyosha Efros

  8. Parametric (global) warping • Examples of parametric warps: aspect rotation translation perspective affine Source: Alyosha Efros

  9. T Parametric (global) warping • Transformation T is a coordinate-changing machine: • p’ = T(p) • What does it mean that T is global? • Is the same for any point p • can be described by just a few numbers (parameters) • Let’s represent T as a matrix: • p’ = Mp p = (x,y) p’ = (x’,y’) Source: Alyosha Efros

  10. Scaling • Scaling a coordinate means multiplying each of its components by a scalar • Uniform scaling means this scalar is the same for all components: 2 Source: Alyosha Efros

  11. X  2,Y  0.5 Scaling • Non-uniform scaling: different scalars per component: Source: Alyosha Efros

  12. Scaling • Scaling operation: • Or, in matrix form: scaling matrix S Source: Alyosha Efros

  13. What transformations can be represented with a 2x2 matrix? 2D Scaling? 2D Rotate around (0,0)? 2D Shear? Source: Alyosha Efros

  14. What transformations can be represented with a 2x2 matrix? 2D Mirror about Y axis? 2D Mirror over (0,0)? 2D Translation? NO! Source: Alyosha Efros

  15. 2D Linear Transformations • Only linear 2D transformations can be represented with a 2x2 matrix. • Linear transformations are combinations of … • Scale, • Rotation, • Shear, and • Mirror Source: Alyosha Efros

  16. Homogeneous Coordinates • Q: How can we represent translation as a 3x3 matrix using homogeneous coordinates? Source: Alyosha Efros

  17. Homogeneous Coordinates • Q: How can we represent translation as a 3x3 matrix using homogeneous coordinates? • A: Using the rightmost column: Source: Alyosha Efros

  18. Translation Homogeneous Coordinates tx = 2ty= 1 Source: Alyosha Efros

  19. Basic 2D Transformations • Basic 2D transformations as 3x3 matrices Translate Scale Rotate Shear Source: Alyosha Efros

  20. 2D Affine Transformations • Affine transformations are combinations of … • Linear transformations, and • Translations • Parallel lines remain parallel

  21. Projective Transformations • Projective transformations: • Affine transformations, and • Projective warps • Parallel lines do not necessarily remain parallel

  22. Recap Linear Transformation Scale, Rotation, Shear, and Mirror Affine Transformation Linear Transformation and Translation Parallel lines remain parallel Projective Transformation Affine transformations, and Projective warps Parallel lines do not necessarily remain parallel

  23. Today • Alignment & warping • 2d transformations • Forward and inverse image warping • Constructing mosaics • Homographies • Robust fitting with RANSAC

  24. Image warping Given a coordinate transform and a source image f(x,y), how do we compute a transformed image g(x’,y’)=f(T(x,y))? T(x,y) y y’ x x’ f(x,y) g(x’,y’) Slide from Alyosha Efros, CMU

  25. Forward warping • Send each pixel f(x,y) to its corresponding location • (x’,y’)=T(x,y) in the second image T(x,y) y y’ x x’ f(x,y) g(x’,y’) Q: what if pixel lands “between” two pixels? Slide from Alyosha Efros, CMU

  26. Forward warping • Send each pixel f(x,y) to its corresponding location • (x’,y’)=T(x,y) in the second image T(x,y) y y’ x x’ f(x,y) g(x’,y’) Q: what if pixel lands “between” two pixels? • A: distribute color among neighboring pixels (x’,y’) • Known as “splatting” Slide from Alyosha Efros, CMU

  27. T-1(x,y) Inverse warping Get each pixel g(x’,y’) from its corresponding location (x,y)=T-1(x’,y’) in the first image y y’ x x x’ f(x,y) g(x’,y’) Q: what if pixel comes from “between” two pixels? Slide from Alyosha Efros, CMU

  28. T-1(x,y) Inverse warping Get each pixel g(x’,y’) from its corresponding location (x,y)=T-1(x’,y’) in the first image y y’ x x x’ f(x,y) g(x’,y’) Q: what if pixel comes from “between” two pixels? • A: Interpolate color value from neighbors • nearest neighbor, bilinear… >> help interp2 Slide from Alyosha Efros, CMU

  29. Bilinear interpolation Sampling at f(x,y): Slide from Alyosha Efros, CMU

  30. Alignment problem • We have previously considered how to fit a model to image evidence • e.g., a line to edge points, or a snake to a deforming contour • In alignment, we will fit the parameters of some transformation according to a set of matching feature pairs (“correspondences”). xi ' xi T

  31. Fitting an affine transformation Affine model approximates perspective projection of planar objects. Figures from David Lowe, ICCV 1999

  32. Fitting an affine transformation • Assuming we know the correspondences, how do we get the transformation?

  33. An aside: Least Squares Example • Say we have a set of data points (X1,X1’), (X2,X2’), (X3,X3’), etc. (e.g. person’s height vs. weight) • We want a nice compact formula (a line) to predict X’s from Xs: Xa + b = X’ • We want to find a and b • How many (X,X’) pairs do we need? • What if the data is noisy? Ax=b overconstrained Source: Alyosha Efros

  34. An aside: Least Squares Example Geometric meaning of LSQ: The columns of A define an n-dimensional subspace in R^m We seek the point Ax, in that subspace, that is closest to b. Q: How do we compute the inverse?

  35. SVD to the rescue Every matrix A (mxn) can be decomposed into: A=USV’ Where: U is mxm, S is mxn and V is nxn. Then pinv(A’A) can be solved as follows: [U,S,V] = svd(A’A); iS = S; iS(iS>1e-6) = 1./iS(iS>1e-6); pinvATA = V*iS*U’; Proof: U*S*V * V’*iS*U’ = U*S*I*iS*U’ = U*I*U’ = I (because U,V are orthonormal)

  36. Geometric meaning of SVD Let A = USV’ Then V is coordinate system of input space and U coordinate system of output space. S is the scaling matrix. Specifically, V’ maps from input space coordinate system to canonical Coordinate system. S scales the coordinates and U maps them to the new Coordinate system. That’s how we should understand: x=pinv(A’A)A’b. Essentially, A’b projects b to the coloumns of A. However, we need to normalize Them and we achieve this by pinv(A’A), which in our case simply scales the Coordinates (because for normal equations A’A then A’A=USU’)

  37. Fitting an affine transformation • Assuming we know the correspondences, how do we get the transformation?

  38. Fitting an affine transformation • How many matches (correspondence pairs) do we need to solve for the transformation parameters? • Once we have solved for the parameters, how do we compute the coordinates of the corresponding point for ?

  39. What are the correspondences? ? • Compare content in local patches, find best matches. • e.g., simplest approach: scan with template, and compute SSD or correlation between list of pixel intensities in the patch • Later in the course: how to select regions according to the geometric changes, and more robust descriptors.

  40. Panoramas . . . image from S. Seitz Obtain a wider angle view by combining multiple images.

  41. How to stitch together a panorama? • Basic Procedure • Take a sequence of images from the same position • Rotate the camera about its optical center • Compute transformation between second image and first • Transform the second image to overlap with the first • Blend the two together to create a mosaic • (If there are more images, repeat) • …but wait, why should this work at all? • What about the 3D geometry of the scene? • Why aren’t we using it? Source: Steve Seitz

  42. p’ H p Homography • How to relate two images from the same camera center? • how to map a pixel from PP1 to PP2? • Think of it as a 2D image warp from one image to another. • A projective transform is a mapping between any two PPs with the same center of projection • rectangle should map to arbitrary quadrilateral • parallel lines aren’t • but must preserve straight lines • called Homography PP2 PP1 Source: Alyosha Efros

  43. Homography p p’ H • To apply a given homography H • Compute p’ = Hp (regular matrix multiply) • Convert p’ from homogeneous to image coordinates

  44. Homography … … To compute thehomography given pairs of corresponding points in the images, we need to set up an equation where the parameters of H are the unknowns…

  45. SVD to the rescue again • We wish to solve Ax=0. • Then Let A=USV’ and take x=V( :, end ) be the last eigenvector corresponding to the smallest eigenvalue of AAT. • Then: • Ax = U*S*V’*V(:,end)Lets look at V’*V(:,end): • V(:,i)’*V(:,end) = 0 for every i<end. • V(:,end)’*V(:,end)=1 • because V is orthonormal. • This is multiplied by S( end, end ) which is the smallest of singular value of A.

  46. Recap: How to stitch together a panorama? • Basic Procedure • Take a sequence of images from the same position • Rotate the camera about its optical center • Compute transformation between second image and first • Transform the second image to overlap with the first • Blend the two together to create a mosaic • (If there are more images, repeat) Source: Steve Seitz

  47. Today • Alignment & warping • 2d transformations • Forward and inverse image warping • Constructing mosaics • Homographies • Robust fitting with RANSAC

  48. Outliers • Outliers can hurt the quality of our parameter estimates, e.g., • an erroneous pair of matching points from two images • an edge point that is noise, or doesn’t belong to the line we are fitting.

  49. Example: least squares line fitting Assuming all the points that belong to a particular line are known

  50. Outliers affect least squares fit

More Related