Image Indexing and Retrieval using Moment Invariants

Image Indexing and Retrieval using Moment Invariants Imran Ahmad School of Computer Science University of Windsor – Canada

Outline • Introduction • Shape-based Retrieval • Image Representation • Moment Invariants • Proposed Approach • Experimental Results • Conclusions I. Ahmad - Windsor, Canada

Introduction • Information Characteristics • Nature • Multiple formats • Complex • Types • Image Database • Time dependent sequence • Video Database I. Ahmad - Windsor, Canada

Introduction (contd.) • Still information • Dynamic information • Temporal Evolution of information • Changes in features and characteristics I. Ahmad - Windsor, Canada

Introduction (contd.) • Image contents • Real world information • Unique features • Need retrieval based on contents I. Ahmad - Windsor, Canada

Image Retrieval • Exact match • Similarity-based retrievals • Color / texture similarity-based retrievals • Spatial similarity-based retrievals • Shape similarity-based retrievals I. Ahmad - Windsor, Canada

Shape-based retrievals • Model-based Object Recognition Approach • Models based on global and local features • Unknown object compared against known ones • Data-Driven Approach • An index for known shapes • Search utilizes such indices I. Ahmad - Windsor, Canada

Image Representation • Images can be defined in terms of: • Global features • Based on overall image composition • Easier to compute • Local features • Based on individual image components • Incorporate spatial information • Computationally expensive I. Ahmad - Windsor, Canada

Shape • How to define a shape? • A geometric property of a figure • Formal definition – independent of language • Described in terms of properties invariant under a group of coordinate transformations • Let is a characteristic function such that For points in the figure Otherwise I. Ahmad - Windsor, Canada

Shape (contd.) • Definition:Let Y be a group of coordinate transformations. The function I is invariant w.r.t. Y if for all characteristic functions and all transformations y e Y • Definition:A shape of a figure is a pair <I, Y>, where I is invariant under the group of coordinate transformations Y. I. Ahmad - Windsor, Canada

Moments • Can capture global information about image • Do not require closed boundaries. • Regular moments – introduced by Hu. • Invariant to translation, rotation and scaling • Algebraic moments • Do not depend on actual values of the coefficients • Central moments • Equivalent to regular moments of an image that has been shifted I. Ahmad - Windsor, Canada

Moments (contd.) • Applications • Image reconstruction • Shape identification such as aircrafts, etc. • Shape recognition • Classifiers I. Ahmad - Windsor, Canada

Moments (contd.) • Let • be the image intensity distribution function • p +qis the order of moments (for p, q =0, 1, 2, …) the algebraic moment of functions are given as: • For a digital image of size M x N I. Ahmad - Windsor, Canada

Moments (contd.) • For centralized moments, we can write: with its digital form as: I. Ahmad - Windsor, Canada

Moments (contd.) • Central moments up to 2nd order are defined as: I. Ahmad - Windsor, Canada

Moments (contd.) I. Ahmad - Windsor, Canada

Moments (contd.) • Algebraic moments by Hu I. Ahmad - Windsor, Canada

Moments (contd.) • Moment Invariants: • Time complexity in computing MI is directly proportional to the number of pixels in the silhouette or forming the boundary. • Let N be the perimeter of the closed boundary • To calculate 2nd order moments, we need: 4(N-1) real additions and 3N real multiplications • To calculate 3rd order moments, we need: 6(N-1) real additions and 12N real multiplications I. Ahmad - Windsor, Canada

Moments (contd.) • A Sample binary image & moment invariants F={0.259179343138514, 0.00801986505055, 0.012354456089699, 0.00827468547136, -0.000000750728194, -0.00005777268349, 0.00000025369430} I. Ahmad - Windsor, Canada

Clustering • Main Categories • Hierarchical methods • A nested sequence of partitions • Involves multiple iterations to cluster objects • Non-hierarchical methods • Assume desired number of clusters at the beginning • Data is reallocated until a particular clustering criteria is optimized. • Objects in a cluster are more similar to each other. I. Ahmad - Windsor, Canada

Clustering (contd.) • K-means clustering • Let a set of N objects in d-dimensional space Rd • k is an integer • Determine a set of k points in Rd, called centers, so as to minimize the mean squared distance from each data point to its nearest center. • Also known as squared-error distortion. I. Ahmad - Windsor, Canada

K-means Clustering (contd.) ALGORITHM: K-means clustering • For pattern vectors P1, P2,…, Pm, set first k pattern vectors to the initial clusters C1=P1, C2=P2, C3=P3,…, Ck=Pk, where m >= k • Assign each pattern vector to the nearest cluster • Compute new cluster means • If new cluster means = old cluster means stop, else go to step 2 I. Ahmad - Windsor, Canada

K-means Clustering (contd.) • K-means Clustering Tree (KCT) • The KCT is a hierarchical data structure similar to a combination of binary search tree and B+ -tree. • Data pointers are stored only at the leaf nodes of tree. • Non-leaf nodes that contain weight vectors. • Non-leaf nodes have links to other nodes. • Unidirectional links • Left child nodes with lesser threshold values • Right child nodes with greater threshold values. I. Ahmad - Windsor, Canada

K-means Clustering (contd.) • K-means Clustering Tree (KCT) – contd. • KCT has a single root node • Leaf nodes of KCT contain features for an image shape and a pointer to the images. • Non-leaf nodes of the tree correspond to the other levels of the index. • The nodes correspond to disk pages and the structure is designed so that search requires visiting only a small number of pages. I. Ahmad - Windsor, Canada

K-means clustering (contd.) • KCT Tree creation Assume 2-D feature vector for each of 19 object as: { {-3.0, 3.0}, {-2.5, 3.0}, {-2.0, 2.0}, {-1.5, 2.0}, {-3.5, 1.5}, {-4.0, 1.0}, {-3.0, 0.5}, {-3.0, 0.0}, {-1.5, 0.5}, { 2.5,-0.5}, { 2.5,-1.0}, { 4.5,-1.0}, { 0.5,-1.5}, { 1.0,-2.0}, { 3.0,-2.0}, { 4.0,-2.0}, { 0.5,-2.5}, { 1.0,-2.5}, { 2.0,-3.0} } I. Ahmad - Windsor, Canada

K-means clustering (contd.) • Same 2-dimensional numerical values • Each object value will be in leaf node of KCT I. Ahmad - Windsor, Canada

K-means clustering (contd.) • Corresponding KCT tree I. Ahmad - Windsor, Canada

K-means Clustering (contd.) • Insertion possibilities • A leaf node is full and a new object is to be inserted in that leaf node. In such case, an insertion results in overflow and, therefore, the node must split. As a result, a new non-leaf node and two leaf nodes are constructed and linked to that part of the tree. We also need to train the non-leaf node to get weight values. • When the node in which the object has to be inserted has only one object in it. In this case, object can be simply added into node without any additional cost. I. Ahmad - Windsor, Canada

K-means Clustering (contd.) • Deletion possibilities • When the sibling node is full-leaf node or non-leaf node and we delete an object from full leaf node  delete object from that node. • When the sibling node is leaf node with an object and we delete an object from full- leaf node  delete object from that node, combine two leaf nodes, delete its parent non-leaf node, and connect remaining full-leaf node to deleted non-leaf node's parent node. I. Ahmad - Windsor, Canada

K-means Clustering (contd.) • Deletion possibilities – contd. • When the sibling node is full-leaf node and we delete an object from leaf node with itself  delete object from that node, delete its parent non-leaf node, and connect remaining full-leaf node to deleted non-leaf node's parent node. • When sibling node is non-leaf node and we delete an object from leaf node with one object  Delete an object from that node, delete its parent non-leaf node, and connect deleted non-leaf node's child node to deleted non-leaf node's parent node. I. Ahmad - Windsor, Canada

K-means Clustering (contd.) Algorithm for retrieving images // Search an object with feature F using KCT • b  block containing root node of a KCT • read block b • while (b is not a leaf node of the KCT) do • next  recall Backpropagation using weights in block b • b  next • read block b • search block b for the most similar object with feature F // search leaf node • if found then • read index file block; display images with object I. Ahmad - Windsor, Canada

Experimental Results • Experimental Setup • Environment: PC with Microsoft Windows 98 • Language: C / C++ • Images • Normalized to grayscale and 128 x 128 • Grayscale images to binary images for chain-codes • Data set size • 100 original images • 5 variants involving translation, rotation and scaling. I. Ahmad - Windsor, Canada

Experimental Results (contd.) • Sample image shapes and their seven moment invariants I. Ahmad - Windsor, Canada

Experimental Results (contd.) A subset of grouping results using database images at the root level of KCT. Objects in top row occupy the left subtree while the bottom row objects become right subtree. I. Ahmad - Windsor, Canada

Experimental Results (contd.) • Grouping results with database images at the 3rd level of KCT. The top row of objects forms the left subtree of KCT while the bottom row is the right subtree. I. Ahmad - Windsor, Canada

Experimental Results (contd.) • Results of sample queries. Query shape is given in row 1, column 1 of each image while the retrieved images include the query shape. I. Ahmad - Windsor, Canada

Experimental Results (contd.) I. Ahmad - Windsor, Canada

Conclusions • Presented a moment invariants based image indexing scheme. • Chain codes are used to reduce size of database • Indexing is based on K-means clustering with k = 2 and Backpropagation to get weights for each node. • Retrieval of images was based on finding the leaf node that includes similar images. • Limitation: Small image collection and limited training data. I. Ahmad - Windsor, Canada

Image Indexing and Retrieval using Moment Invariants