1 / 24

Motion from image and inertial measurements (additional slides)

This presentation by Dennis Strelow from Carnegie Mellon University explores robust image feature tracking using the Lucas-Kanade method and introduces the Smalls tracker, specifically designed for six degrees of freedom (6 DOF) motion estimation. It emphasizes the importance of accurate feature tracking, the integration of image and inertial measurement data, and the challenges posed by various real-world sequences. Key concepts include epipolar geometry estimation, SIFT-based matching, and innovative heuristics to enhance feature tracking performance, overcoming limitations of traditional methods.

kevyn
Télécharger la présentation

Motion from image and inertial measurements (additional slides)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Motion from image and inertial measurements (additional slides) Dennis Strelow Carnegie Mellon University

  2. Outline Robust image feature tracking (in detail) Lucas-Kanade and real sequences The “smalls” tracker Motion from omnidirectional images Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 20052

  3. Robust image feature tracking: Lucas-Kanade and real sequences (1) • Combining image and inertial measurements improves our situation, but… • we still need accurate feature tracking tracking • some sequences do not come with inertial measurements Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 20053

  4. Robust image feature tracking: Lucas-Kanade and real sequences (2) • better feature tracking for improved 6 DOF motion estimation • remaining results will be image-only Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 20054

  5. Robust image feature tracking: Lucas-Kanade and real sequences (3) • Lucas-Kanade has been the go-to feature tracker for shape-from-motion • minimizes a correlation-like matching error • using general minimization • evaluates the matching error at only a few locations • subpixel resolution Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 20055

  6. Robust image feature tracking: Lucas-Kanade and real sequences (4) Additional heuristics used to apply Lucas-Kanade to shape-from-motion: Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 20056

  7. Robust image feature tracking: Lucas-Kanade and real sequences (5) But Lucas-Kanade performs poorly on many real sequences… Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 20057

  8. Robust image feature tracking: the “smalls” tracker (1) • smalls is a new feature tracker targeted at 6 DOF motion estimation • exploits the rigid scene assumption • eliminates the heuristics normally used with Lucas-Kanade • SIFT is an enabling technology here Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 20058

  9. Robust image feature tracking: the “smalls” tracker (2) • First step: epipolar geometry estimation • use SIFT to establish matches between the two images • get the 6 DOF camera motion between the two images • get the epipolar geometry relating the two images Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 20059

  10. Robust image feature tracking: the “smalls” tracker (3) Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200510

  11. Robust image feature tracking: the “smalls” tracker (4) Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200511

  12. Robust image feature tracking: the “smalls” tracker (5) • Second step: track along epipolar lines • use nearby SIFT matches to get initial position on epipolar line • exploits the rigid scene assumption • eliminates heuristic: pyramid Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200512

  13. Robust image feature tracking: the “smalls” tracker (6) • Third step: prune features • geometrically inconsistent features are marked as mistracked and removed • clumped features are pruned • eliminates heuristic: detecting mistracked features based on convergence, error Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200513

  14. Robust image feature tracking: the “smalls” tracker (7) • Fourth step: extract new features • spatial image coverage is the main criterion • required texture is minimal when tracking is restricted to the epipolar lines • eliminates heuristic: extracting only textured features Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200514

  15. Robust image feature tracking: the “smalls” tracker (8) Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200515

  16. Robust image feature tracking: the “smalls” tracker (9) left: odometry only right: images only • average error: 1.74 m • maximum error: 5.14 m • total distance: 230 m Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200516

  17. Robust image feature tracking: the “smalls” tracker (10) • Recap: • exploits the rigid scene and eliminates heuristics • allows hands-free tracking for real sequences • can still be defeated by textureless areas or repetitive texture Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200517

  18. Outline Robust image feature tracking (in detail) Motion from omnidirectional images Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200518

  19. Motion from omnidirectional images (1) Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200519

  20. Motion from omnidirectional images (2) Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200520

  21. Motion from omnidirectional images (3) Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200521

  22. Motion from omnidirectional images (4) Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200522

  23. Motion from omnidirectional images (5) left: non-rigid camera right: rigid camera squares: ground truth points solid: image-only estimates dash-dotted: image-and-inertial estimates Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200523

  24. Motion from omnidirectional images (6) • In this experiment: • omni images • conventional images + inertial • have roughly the same advantages • But in general: • inertial has some advantages that omni images alone can’t produce • omni images can be harder to use Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 200524

More Related