1 / 22

Semi-Local Affine Parts for 3D Object Recognition: A Weakly Supervised Learning Approach

This research presents a method for recognizing 3D object classes by utilizing semi-local affine parts. It addresses challenges such as geometric invariance, robustness to clutter and occlusion, and weakly supervised learning. By employing local affine regions detected via the Laplacian detector, the work proposes a two-image matching technique based on geometric and photometric consistency to establish correspondences. It also highlights the validation of candidate parts using training images. The framework is evaluated on butterfly species with promising results and sets the stage for future work in recognizing non-rigid object categories.

phila
Télécharger la présentation

Semi-Local Affine Parts for 3D Object Recognition: A Weakly Supervised Learning Approach

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Semi-Local Affine Partsfor Object Recognition Svetlana Lazebnik, Jean PonceUniversity of Illinois at Urbana-Champaign Cordelia Schmid INRIA Rhône-Alpes BMVC 2004

  2. Overview • Goal: • Learning models for recognition of 3D object classes • Challenges: • Geometric invariance • Robustness to clutter, occlusion • Weakly supervised learning • Proposed approach: • An object representation using semi-local affine parts

  3. Low-Level Features: Local Affine Regions • This work: Laplacian detector (Gårding & Lindeberg, 1996) • Other detectors: Kadir et al. (2004), Matas et al. (2002), Mikolajczyk & Schmid (2002), Tuytelaars & Van Gool (2004), etc.

  4. In practice: two-image matching followed by validation initial pair validation set candidate part Learning Parts • Ideal approach: simultaneous correspondence search across entire training set

  5. Two-Image Matching • Goal: to find collections of local affine regions that can be mapped onto each other using a single affine transformation • Implementation: greedy search based on geometric and photometric consistency constraints • Returns multiple correspondence hypotheses • Automatically determines number of regions in correspondence • Works on unsegmented, cluttered images (weakly supervised learning) A

  6. Matching: Details • Initialization: • Identify triples of neighboring regions (i, j, k) in first image • Find all triples (i', j', k') in the second image such that i' (resp. j', k') is a potential match of i (resp. j, k), and j', k' are neighbors of i' j j' i i' k' k

  7. Matching (cont.) • Beginning with each seed triple, iterate: • Estimate the affine transformation between centers of corresponding regions in current group of matches A

  8. Matching (cont.) • Beginning with each seed triple, iterate: • Estimate the affine transformation between centers of corresponding regions in current group of matches • Determine geometric consistency of current group of matches • Geometric consistency criteria: • Distance between ellipse centers (residual) • Difference of major and minor axis lengths • Difference of ellipse orientations

  9. Matching (cont.) • Beginning with each seed triple, iterate: • Estimate the affine transformation between centers of corresponding regions in current group of matches • Determine geometric consistency of current group of matches • Search for additional matches in the neighborhood of the current group

  10. Matching: 3D Objects

  11. Matching: 3D Objects closeup closeup

  12. Matching: Faces spurious match ???

  13. Finding Repeated Patterns and Symmetries

  14. Learning Object Models for Recognition • Match multiple pairs of training images to produce a set of candidate parts • Use additional validation images to evaluate repeatability of parts and individual regions • Retain a fixed number of parts having the best repeatability score

  15. Recognition Experiment: Butterflies Admiral Swallowtail Machaon Monarch 1 Monarch 2 Peacock Zebra • 26 training images per class • 8 initial pairs • 10 validation images • 437 test images • 619 images total

  16. Butterfly Parts

  17. Recognition • Top 10 parts per class used for recognition • Relative repeatability score: • Classification results: total number of regions detectedtotal part size Total part size (smallest/largest)

  18. Classification Rate vs. Number of Parts

  19. Detection Results (ROC Curves) Circles: reference relative repeatability rates. Red square: ROC equal error rate (in parentheses)

  20. Successful Detection Examples Training images Test images (blue: occluded regions) All regions found in the test images

  21. Unsuccessful Detection Examples Training images Test images (blue: occluded regions) All regions found in the test images

  22. Future Work • Goal: • Recognize highly variable, non-rigid object categories • Proposed approach: • Treat semi-local affine parts as “black boxes” • Model spatial relations between parts • Learn these relations from training data in a weakly supervised fashion

More Related