1 / 24

Bag of Features Approach: recent work, using geometric information

This work explores recent advancements in object recognition and retrieval from extensive image collections, emphasizing geometric information. Key challenges addressed include object category and specific object recognition, and methods for improving efficiency using geometric constraints and bag-of-features (BoF) representation. By integrating local features and spatial verification strategies, the research builds upon existing frameworks and suggests new directions to optimize indexing and search algorithms. The findings highlight the potential of geometry-preserving techniques and the need for further exploration in cross-training and dataset dependency.

rock
Télécharger la présentation

Bag of Features Approach: recent work, using geometric information

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Bag of Features Approach: recent work, using geometric information

  2. Problem • Search for object occurrences in very large image collection

  3. 2 sub problems • Object Category Recognition and Specific Object Recognition

  4. Motivation • Look for product information • Look for similar products

  5. Related work on large scale image search • Most systems build upon the BoF framework [Sivic & Zisserman 03] • Large (hierarchical) vocabularies [NisterStewenius 06] • Improveddescriptorrepresentation [Jégou et al 08, Philbin et al 08] • Geometry used in index [Jégou et al 08, Perdoc’h et al 09] • Query expansion [Chum et al 07] • … • Efficiency improved by: • Min-hash and Geometrical min-hash [Chum et al. 07-09] • Compressing the BoF representation [Jégou et al. 09]

  6. Local Features - SIFT

  7. Creating a visual vocabulary 1 2 3 4

  8. Inverted Index Index construction Searching

  9. Use geometry • Possible directions: • Change/optimize spatial verification stage • Insert a new geometric information to the index • Ordered BOF • Bundled features • Visual phrases • Change the searching algorithm

  10. Survey for today • Spatial Bag-of-features [Cao, CVPR2010] • Image Retrieval with Geometry-Preserving Visual Phrases [Zhang Jia Chen, CVPR2011] • Smooth Object Retrieval using a Bag of Boundaries [ArandjeloviZisserman, ICCV2011]

  11. Spatial BOF • Basic idea:

  12. Spatial BOF • Constructing linear and circular ordered bag-of-features:

  13. Spatial BOF • Translation invariance:

  14. Spatial BOF • Pros: • Gets better performance than BOF+RANSAC for large scale dataset* • Same format as standard BOF • Cons: • Is dataset dependent because of need of training • Do not present the results for large scale dataset with transfer learning from another dataset • Future work • Check it with cross training for large dataset. Otherwise, it is not worth working further.

  15. Geometry-Preserving Visual Phrases • Basic idea:

  16. Geometry-Preserving Visual Phrases • Representation • Quantize image to 10x10 grid • Histogram of GVPs of length k • GVP dictionary size is “choose k from N visual words”

  17. Geometry-Preserving Visual Phrases • Pros: • Outperforms BOV + RANSAC • Cons: • Only translation invariant because of memory • Future work

  18. BOF for smooth objects Idea: Gradient Segment The information used for retrieval Query object

  19. BOF for smooth objects Results:

  20. BOF for smooth objects Segmentation phase • Over segmentation with super-pixels • Classification of super-pixels: • 3208 feature vector (median(Mag(Grad)), 4 bits, color histogram, BOF) • SVM • Post-processing

  21. BOF for smooth objects Boundary description phase: • Sample points on the boundary • Calculate HoG at each point in 3 scales 340 dimensional L2 normalized vector * The descriptor is not rotation invariant

  22. BOF for smooth objects Retrieval procedure: • Boundary descripors are quantized (k=10k) • Standard BOF scheme* • Spatial verification for top 200 with loose affine homography (errors up to 100pixs) * No spatial information is recorded in the histogram

  23. BOF for smooth objects • Pros: • Solves the smooth object retrieval problem • Fast • Cons: • Is dataset dependent because of need of training • Limited to objects with “solid” materials – segmentation has to catch the object’s boundary • Future work • Eliminate the training step

  24. Summary • There is an active research in the field of CBIR to exploit geometry information. • Each method with its limitations • Still no widely accepted solution • Like spatial verification with RANSAC

More Related