A Scheme for Visual Feature based Image Indexing

1. A Scheme for Visual Feature based Image Indexing HongJiang Zhang and Di Zhong SPIE Conf. on Storage and Retrieval for Image and Video Databases Feb 1995 Presented by Vibhore Vardhan

2. Motivation • Digital images need to be manipulated and managed as images • Retrieve visual data based on visual content of image • E.g.: IBM’s QBIC system • More emphasis on deriving visual features such as color, texture • Need an effective indexing scheme utilizing these features • Necessary to browse large image databases

3. Multi-dimensional Index • Pre-computed visual features for each item in database • Key attribute for an item is a feature vector • Search based on similarities between feature vectors • Three popular approaches to multi-dimensional indexing: • R* tree, Linear quadtrees, and Grid files • But they assume the following: • Distance = Euclidean distance of points in feature space • Dimensionality of feature space is low • Efficient filter allows false positives, but no false dismissals • Lower dimensional feature space or narrow search space

4. Tree Indexing by Abstraction and Classification • For given attribute Aj, identify and label all objects that share Aj • Identical value or a certain range al • Objects with same label are clustered to form an abstraction • Abstractions are represented as nodes in the index tree • Apply these abstraction operations recursively to reach root node • Automate using Self-Organization feature Maps (SOM) • Unsupervised learning based on a grid of artificial neurons • Weights are adapted to match input vectors in a training set • First described by Teuvo Kohonen

5. Architecture of Self-Organization Map # of nodes > # of possible classes associated weight image feature vector • Two layer SOM – mapping from input data in Rn onto a 2-d grid • All ref. vectors compared with 1 input vector according to metrics • Select best matching node in the map, update neighbors • After several iterations, SOM adapts to input

6. Hierarchical SOM • Modify SOM to meet certain properties • Constructs an index tree which forms similarity space of feature data • First form bottom level L through learning • Each node in L represents a group of image which are similar • Higher levels are created by applying clustering and projection

7. Results – Texture Features based Index Iconic map of Brodatz texture database

8. Results – Texture Features based Index

9. Results – Texture Features based Index • Texture feature set model (20 dimension feature vector): • Multi-resolution simultaneous auto-regressive (MRSAR) • Combined MRSAR with coarseness features and gray histograms • Improves accuracy – feature vector size goes up to 30 • Evaluated on Brodatz texture database • 112 classes of images (512x512 8-bits) • 9 subimages (128x128 8-bits) in each class • Retrieval rate = number of retrieved subimages (same class as query) • number of retrieval neighbors

10. Results – Texture Features based Index • MRSAR does as well as global search for 9-nearest neighbor searching • 5x speed improvement • Adding coarseness and histogram improves accuracy by 6%

11. Results – Color Histogram based Index • Color histogram of images as feature vector for 317 images • 106 images classified into 7 categories, rest act as noise • Compared indexing using color histograms in 3 color spaces • Results for RGB space with 10 neighbors: Retrieval accuracy similar to global search, but faster LUV color space gave the highest retrieval rate

12. Conclusion • Initial work in developing an effective indexing scheme • Feature vector for images have high dimensions • Not suitable for traditional indexing approaches • Implements hierarchical SOM • Results show good accuracy and speedups

13. DEMO • PicSOM