1 / 74

Binocular Stereo #1

Binocular Stereo #1. Topics. 1. Principle 2. binocular stereo basic equation 3. epipolar line 4. features and strategies for matching. single image is ambiguous. A. a”. a’. another image taken from a different direction gives the unique 3D point. Binocular stereo. Base line.

perry
Télécharger la présentation

Binocular Stereo #1

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Binocular Stereo #1

  2. Topics 1. Principle 2. binocular stereo basic equation 3. epipolar line 4. features and strategies for matching

  3. single image is ambiguous A a” a’ another image taken from a different direction gives the unique 3D point Binocular stereo

  4. Base line Epipolar line One image point Possible line of sight Epipolar plane Epipolar line constraints Corresponding points lie on the Epipolar lines Epipolar line constratints

  5. C2 C1 Epipolar geometry (multiple points) e2 e1 • Epipoles: • intersections of baseline with image planes • projection of the optical center in another image • the vanishing points of camera motion direction

  6. Examples of epipolar geometry

  7. Examples of epipolar geometry

  8. Examples of epipolar geometry

  9. rectification Characteristics of epipolar line

  10. Basic binocular stereo equation A physical point left image point right image point right image plane left image plane focal length right image center z left image center World coordinate system base line length

  11. Camera Model Pinhole camera

  12. y x (X, Y, Z) (x, y) (sX, sY, sZ) Image plane Y X -Z Camera Model geometry Perspective projection View point (Optical center) f : focal length

  13. d + x d - x Basic binocular stereo equation z=-2df/(x”-x’) x”-x’: disparity 2d : base line length -z x’ x” f d d z

  14. Classic algorithms for binocular Stereo MIT group Marr-Poggio Marr-Poggio-Grimson Nishihara-Poggio Lucas-Kanade Ohta-Kanade Matthie-Kanade Okutomi-Kanade Baker Hannah Moravec Barnard-Thompson CMU group Stanford group

  15. Features for matching 10 11 12 10 11 12 10 11 12 10 11 12 11 15 16 a. brightness b. edges c. edge intervals d. interest points

  16. Strategies for matching 10 10 10 10 5 10 10 10 10 10 10 10 10 5 10 10 10 10 10 10 10 10 10 10 10 10 10 a. relaxation b. coarse to fine c. dynamic programming global optimam local optimam local optimam

  17. Main purpose of development Marr-Poggio Marr-Poggio-Grimson Nishihara-Poggio Lucas-Kanade Ohta-Kanade Matthie-Kanade Okutomi-Kanade Baker Hannah Moravec Barnard-Thompson simulate human stereo simulate human stereo map making map making map making map making map making navigation navigation navigation

  18. Features for matching Marr-Poggio Marr-Poggio-Grimson Nishihara-Poggio Lucas-Kanade Ohta-Kanade Matthie-Kanade Okutomi-Kanade Baker Hannah Moravec Barnard-Thompson points(random dots) edges intervals brightness(gradient) intervals brightness brightness edges interest points interest points interest points

  19. Strategies for matching Marr-Poggio Marr-Poggio-Grimson Nishihara-Poggio Lucas-Kanade Ohta-Kanade Matthie-Kanade Okutomi-Kanade Baker Hannah Moravec Barnard-Thompson relaxation coarse to fine coarse to fine relaxation dynamic programming Relaxation(Kalman filter) relaxation dynamic programming coarse to fine coarse to fine relaxation

  20. Summary • 1.binocular stereo takes two images of 3D point from two different positions and determines its 3D coordinate system. • 2. Epipolar line • 2D matching • ↓ • 1D matching • 3. Features for matching • ---brightness,edges,edge interval,interest point • 4. Strategies for matching • ---relaxation,coarse to fine,dynamic programming • 5. Read • B&B pp.88-93 • Horn pp.299-303

  21. Binocular Stereo #2

  22. Topics case study area-based stereo Marr-poggio stereo simulate human visual system Ohta-Kanade stereo aerial image analysis Moravec stereo navigation

  23. Classification of stereo method 1. Features for matching a. brightness value b. point c. edge d. region 2. Strategies for matching a. brute-force (not a strategy ???) b. coarse-to-fine c. relaxation d. dynamic programming 3. Constraints for matching a. epipolar lines b. disparity limit c. continuity d. uniqueness

  24. Area-based stereo 1. method b c • 2. problem • a. trade-off of window size and resolution • b. dull peak b c b c

  25. Area-based stereo 1. Features for matching a. brightness value b. point c. edge d. region 2. Strategies for matching a. brute-force (not a strategy ???) b. coarse-to-fine c. relaxation d. dynamic programming 3. Constraints for matching a. epipolar lines b. disparity limit c. continuity d. uniqueness

  26. Marr-Poggio Stereo(`76) Simulating human visual system (random dot stereo gram) Marr,Poggio “Coopertive computation of stereo disparity” Science 194,283-287

  27. Input : random dot stereo left image random dot shift the catch pat right image we can see the height different between the central and peripheral area

  28. Constraints • Epipolar line constraint • Uniqueness constraint • each point in a image has only one depth value O.K.No. • Continuity constraint • each point is almost sure to have a depth value near the values of neighbors O.K. No.

  29. D E F A B C A B D E F C Uniqueness constraint prohibits two or more matching points on one horizontal or vertical lines continuity constraint attracts more matching on a diagonal line (E-A) A B C (E-B) prohibit (E-C) (D-A) attract (E-B) Same depth (F-C) attract

  30. relaxation 10 10 10 10 5 10 10 10 10 10 10 10 10 5 10 10 10 10 10 10 10 10 10 10 10 10 10 n n+1

  31. Marr-Poggio Stereo(`76) 1. Features for matching a. brightness value b. point c. edge d. region 2. Strategies for matching a. brute-force (not a strategy ???) b. coarse-to-fine c. relaxation d. dynamic programming 3. Constraints for matching a. epipolar lines b. disparity limit c. continuity d. uniqueness simulate the human visual system (MIT)

  32. Ohta-Kanade Stereo(`85) Map making Ohta,Kanade “Stereo by intra- and inter-scanline search using dynamic programming” ,IEEE Trans.,Vol. PAMI-7,No.2,pp.139-14

  33. now matching become 1D to 1D L1 L2 L3 L4 L5 L6 R1 R2 R3 R4 R5 R6 L disparity R yet, N line * ML * MR (512 * 100 * 100 * 10 m sec = 15 hours)

  34. Path Search • Matching problem can be considered as a path search problem • define a cost at each candidate of path segment based some ad-hoc function 10 100 100

  35. Dynamic programming We can formalize the path finding problem as the following iterative formula optimum cost to K cost between M and K 3 0 2 1 Optimum costs are known

  36. stereo pair edges

  37. path disparity depth

  38. stereo pair edges depth

  39. Ohta-Kanade Stereo(`85) 1. Features for matching a. brightness value b. point c. edge d. region 2. Strategies for matching a. brute-force (not a strategy ???) b. coarse-to-fine c. relaxation d. dynamic programming 3. Constraints for matching a. epipolar lines b. disparity limit c. continuity d. uniqueness aerial image analysis (CMU) Brightness of interval

  40. Moravec Stereo(`79) navigation Moravec “Visual mapping by a robot rover” Proc 6th IJCAI,pp.598-600 (1979)

  41. Moravec’s cart Slide stereo Motion stereo

  42. Slider stereo (9 eyes stereo) • 9C2 = 36 stereo pairs!!! • each stereo has an uncertainty measure • uncertainty = 1 / base-line • each stereo has a confidence measure long base line large uncertainty

  43. matching expand matching expand matching Coarse to fine

  44. area:confidence measure σ estimated distance σ:uncertainty measure Interest point 9C2 = 36 curves

  45. Moravec Stereo(`81) 1. Features for matching a. brightness value b. point c. edge d. region 2. Strategies for matching a. brute-force (not a strategy ???) b. coarse-to-fine c. relaxation d. dynamic programming 3. Constraints for matching a. epipolar lines b. disparity limit c. continuity d. uniqueness navigation (Stanford) interest point

  46. Summary 1. Two images from two different positions give depth information 2. Epipolar line and plane 3. Basic equation Z=-2df/(x”-x’) x”-x’: disparity 2d : base line length 4. case study area-based stereo Marr-poggio stereo simulate human visual system Ohta-Kanade stereo aerial image analysis Moravec stereo navigation 5. Read Horn pp.299-303

  47. F matrix

  48. Camera Model Pinhole camera

More Related