1 / 44

Structure from Motion: Calibration, Perspective Geometry, and System Building Overview

This lecture delves into the intricacies of Structure from Motion (SfM) as presented by renowned speakers Sebastian Thrun, Rick Szeliski, Hendrik Dahlkamp, and Dan Morris at Stanford University. It covers essential topics such as camera calibration with unknown targets, the principles of perspective geometry, and strategies for building robust systems utilizing stereo or laser ranging. The discussion also touches on correspondence establishment, nonlinear optimization through bundle adjustment, and various methods for reconstructing 3D structures and camera motion from multiple images.

kaspar
Télécharger la présentation

Structure from Motion: Calibration, Perspective Geometry, and System Building Overview

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Stanford CS223B Computer Vision, Winter 2005Lecture 11: Structure From Motion 2 Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and Dan Morris, Stanford

  2. Overall Distribution

  3. Question 1: Calibration

  4. Question 1: Calibration • Calibration with planar unknown target • Unknown parameters • 4 intrinsics • 6K extrinsics (K = #images) • 2M calibration target parameters (but can’t recover 3) • 2KM constraints

  5. Question 2: Perspective Geometry

  6. Question 2: Perspective Geometry • Collinearity in 3D  2D (but not converse) • Order in 3D  2D (but not converse) • Equidistance: Not preserved! • Proof (collinearity in 2D):

  7. Question 3: Stereopsis

  8. Question 3: Stereopsis

  9. Question 3: Stereopsis • How does DZ scale with Z? – in approximation!!!

  10. Question 4: True or False

  11. Question 5: Build A System!

  12. Question 5: Build A System! • Range: stereo or laser • Classification : template, optical flow?, SIFT? • Alternatively: segmentation, range discontinuities • Prediction: person and car • Robustness: normalize image, bring light source • (many other possibilities)

  13. Stanford CS223B Computer Vision, Winter 2005Lecture 11: Structure From Motion 2 Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and Dan Morris, Stanford

  14. Structure From Motion (1) [Tomasi & Kanade 92]

  15. Structure From Motion (2) [Tomasi & Kanade 92]

  16. Structure From Motion (3) [Tomasi & Kanade 92]

  17. Structure From Motion • Problem 1: • Given n points pij =(xij, yij) in m images • Reconstruct structure: 3-D locations Pj =(xj, yj, zj) • Reconstruct camera positions (extrinsics) Mi=(Aj, bj) • Problem 2: • Establish correspondence: c(pij)

  18. The “Trick Of The Day” • Replace Euclidean Geometry by Affine Geometry • Solve SFM linearly (“closed” form) • Post-Process to make Euclidean • By Tomasi and Kanade, 1992

  19. Orthographic Camera Model Extrinsic Parameters Rotation Orthographic Projection Limit of Pinhole Model:

  20. Orthographic Projection Limit of Pinhole Model: Orthographic Projection

  21. The Affine SFM Problem

  22. Count # Constraints vs #Unknowns • m camera poses • n points • 2mn point constraints • 8m+3n unknowns • Suggests: need 2mn  8m + 3n • But: Can we really recover all parameters???

  23. How Many Parameters Can’t We Recover? We can recover all but… Place Your Bet!

  24. The Answer is (at least): 12

  25. Points for Solving Affine SFM Problem • m camera poses • n points • Need to have: 2mn  8m + 3n-12

  26. Affine SFM Fix coordinate system by making p0=origin Rank Theorem: Q has rank 3 Proof:

  27. The Rank Theorem 2m elements n elements

  28. Tomasi/Kanade 1992 Singular Value Decomposition

  29. Tomasi/Kanade 1992 Gives also the optimal affine reconstruction under noise

  30. Back To Orthographic Projection Find C and d for which constraints are met

  31. Back To Projective Geometry Orthographic (in the limit) Projective

  32. The “Trick Of The Day” • Replace Euclidean Geometry by Affine Geometry • Solve SFM linearly (“closed” form) • Post-Process to make Euclidean • By Tomasi and Kanade, 1992

  33. SFM With Projective Camera: See Rick Szeliski’s Lecture! Non-Linear Optimization Problem: Bundle Adjustment!

  34. Structure From Motion • Problem 1: • Given n points pij =(xij, yij) in m images • Reconstruct structure: 3-D locations Pj =(xj, yj, zj) • Reconstruct camera positions (extrinsics) Mi=(Aj, bj) • Problem 2: • Establish correspondence: c(pij)

  35. The Correspondence Problem View 1 View 2 View 3

  36. Correspondence: Solution 1 • Track features (e.g., optical flow) • …but fails when images taken from widely different poses

  37. Correspondence: Solution 2 • Start with random solution A, b, P • Compute soft correspondence: p(c|A,b,P) • Plug soft correspondence into SFM • Reiterate • See Dellaert et al 2003, Machine Learning Journal

  38. Example

  39. Results: Cube

  40. Animation

  41. Tomasi’s Benchmark Problem

  42. Reconstruction with EM

  43. 3-D Structure

  44. Summary SFM • Problem • Determine feature locations (=structure) • Determine camera extrinsic (=motion) • The name SFM is somewhat of a misdemeanor • Two Principal Solutions • Nonlinear optimization (local minima) • Linear (affine geometry) • Correspondence • RANSAC • Expectation Maximization

More Related