madelia
Uploaded by
41 SLIDES
545 VUES
410LIKES

Overview of Projective Structure from Motion Techniques and Algorithms

DESCRIPTION

This document provides a comprehensive overview of Projective Structure from Motion (SfM), a fundamental concept in computer vision and photogrammetry. It discusses affine and orthographic projections, singular value decomposition, and the key equations relating to the reconstruction of 3D structures from 2D images. The projective ambiguity and the importance of non-linear least squares optimization are highlighted, alongside algorithms for motion estimation and bundle adjustment. This material serves as an essential guide for anyone looking to deepen their understanding of 3D reconstruction from multiple perspective images.

1 / 41

Télécharger la présentation

Overview of Projective Structure from Motion Techniques and Algorithms

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Projective structure from motion Marc Pollefeys COMP 256 Some slides and illustrations from J. Ponce, …

  2. Last class: Affine camera • The affine projection equations are how to find the origin? or for that matter a 3D reference point? affine projection preserves center of gravity

  3. All equations can be collected for all i and j • where Orthographic factorization • The orthographic projection equations are • where Note that P and X are resp. 2mx3 and 3xn matrices and therefore the rank of x is at most 3

  4. Orthographic factorization • Factorize m through singular value decomposition • An affine reconstruction is obtained as follows Closest rank-3 approximation yields MLE!

  5. Tentative class schedule

  6. PROJECTIVE STRUCTURE FROM MOTION • The Projective Structure from Motion Problem • Elements of Projective Geometry • Projective Structure and Motion from Two Images • Projective Motion from Fundamental Matrices • Projective Structure and Motion from Multiple Images Reading: Chapter 13.

  7. Problem: estimate the m 3x4 matrices M and the n positions P from the mn correspondences p . i j ij The ProjectiveStructure-from-Motion Problem Given m perspective images of n fixed points P we can write j 2mn equations in 11m+3n unknowns Overconstrained problem, that can be solved using (non-linear) least squares!

  8. If M and P are solutions, i j So are M’ and P’ where i j The Projective Ambiguity of Projective SFM When the intrinsic and extrinsic parameters are unknown and Q is an arbitrary non-singular 4x4 matrix. Q is aprojectivetransformation.

  9. Projective Spaces: (Semi-Formal) Definition

  10. A Model of P( R ) 3

  11. Projective Subspaces and Projective Coordinates

  12. Projective Subspaces and Projective Coordinates P Projective coordinates

  13. Projective Subspaces Given a choice of coordinate frame Line: Plane:

  14. Affine and Projective Spaces

  15. Affine and Projective Coordinates

  16. sin(+)sin(+) {A,B;C,D}= sin(++)sin Cross-Ratios Collinear points Pencil of planes Pencil of coplanar lines

  17. Cross-Ratios and Projective Coordinates * Along a line equipped with the basis * In a plane equipped with the basis In 3-space equipped with the basis

  18. Projective Transformations Bijective linear map: Projective transformation: ( = homography ) Projective transformations map projective subspaces onto projective subspaces and preserve projective coordinates. Projective transformations map lines onto lines and preserve cross-ratios.

  19. Perspective Projections induce projective transformations between planes.

  20. Geometric Scene Reconstruction Idea: use (A,O”,O’,B,C) as a projective basis.

  21. Reprinted from “Relative Stereo and Motion Reconstruction,” by J. Ponce, T.A. Cass and D.H. Marimont, Tech. Report UIUC-BI-AI-RCV-93-07, Beckman Institute, Univ. of Illinois (1993).

  22. Motion estimation from fundamental matrices Q Once M and M’ are known, P can be computed with LLS. Facts: b’ can be found using LLS.

  23. Projective Structure from Motion and Factorization Factorization?? • Algorithm (Sturm and Triggs, 1996) • Guess the depths; • Factorize D; • Iterate. Does it converge? (Mahamud and Hebert, 2000)

  24. Bundle adjustment - refining structure and motion • Minimize reprojection error • Maximum Likelyhood Estimation (if error zero-mean Gaussian noise) • Huge problem but can be solved efficiently (exploit sparseness)

  25. Bundle adjustment • Developed in photogrammetry in 50´s

  26. Non-linear least squares • Linear approximation of residual • allows quadratic approximation of sum- of-squares Minimization corresponds to finding zeros of derivative N Levenberg-Marquardt: extra term to deal with singular N (decrease/increase l if success/failure to descent) (extra term = descent term)

  27. P1 P2 P3 M U1 U2 W U3 WT V 3xn (in general much larger) 12xm Bundle adjustment • Jacobian of has sparse block structure • cameras independent of other cameras, • points independent of other points im.pts. view 1 Needed for non-linear minimization

  28. U-WV-1WT WT V 3xn 11xm Bundle adjustment Eliminate dependence of camera/motion parameters on structure parameters Note in general 3n >> 11m Allows much more efficient computations e.g. 100 views,10000 points, solve 1000x1000, not 30000x30000 Often still band diagonal use sparse linear algebra algorithms

  29. Sequential SfM • Initialize motion from two images • Initialize structure • For each additional view • Determine pose of camera • Refine and extend structure • Refine structure and motion

  30. (reference plane;arbitrary) Initial projective camera motion • Choose P and P´compatible with F Reconstruction up to projective ambiguity • Same for more views? • Initialize motion • Initialize structure • For each additional view • Determine pose of camera • Refine and extend structure • Refine structure and motion different projective basis

  31. Initializing projective structure • Reconstruct matches in projective frame by minimizing the reprojection error • Initialize motion • Initialize structure • For each additional view • Determine pose of camera • Refine and extend structure • Refine structure and motion Non-iterative optimal solution

  32. Projective pose estimation • Infere 2D-3D matches from 2D-2D matches • Compute pose from (RANSAC,6pts) X F x • Initialize motion • Initialize structure • For each additional view • Determine pose of camera • Refine and extend structure • Refine structure and motion Inliers:

  33. Refining and extending structure • Refining structure • Extending structure 2-view triangulation (Iterative linear) • Initialize motion • Initialize structure • For each additional view • Determine pose of camera • Refine and extend structure • Refine structure and motion

  34. Refining structure and motion • use bundle adjustment Also model radial distortion to avoid bias!

  35. Hierarchical structure and motion recovery • Compute 2-view • Compute 3-view • Stitch 3-view reconstructions • Merge and refine reconstruction F T H PM

  36. Metric structure and motion use self-calibration (see next class) Note that a fundamental problem of the uncalibrated approach is that it fails if a purely planar scene is observed (in one or more views) (solution possible based on model selection)

  37. Dealing with dominant planes

  38. PPPgric HHgric

  39. Farmhouse 3D models (note: reconstruction much larger than camera field-of-view)

  40. Application: video augmentation

  41. Next class: Camera calibration (and self-calibration) Reading: Chapter 2 and 3

More Related