1 / 29

An Unsupervised Learning Approach to Content-Based Image Retrieval

An Unsupervised Learning Approach to Content-Based Image Retrieval. Yixin Chen & James Z. Wang The Pennsylvania State University. Outline. Introduction Cluster-based retrieval of images Experiments Conclusions and future work. Image Retrieval. The driving forces Internet

ull
Télécharger la présentation

An Unsupervised Learning Approach to Content-Based Image Retrieval

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. An Unsupervised Learning Approach to Content-Based Image Retrieval Yixin Chen & James Z. Wang The Pennsylvania State University IEEE Int'l Symposium on Signal Processing and its Applications

  2. Outline • Introduction • Cluster-based retrieval of images • Experiments • Conclusions and future work IEEE Int'l Symposium on Signal Processing and its Applications

  3. Image Retrieval • The driving forces • Internet • Storage devices • Computing power • Two approaches • Text-based approach • Content-based approach IEEE Int'l Symposium on Signal Processing and its Applications

  4. Text-Based Approach • Input keywords descriptions Text-Based Image Retrieval System Elephants Image Database IEEE Int'l Symposium on Signal Processing and its Applications

  5. Text-Based Approach • Index images using keywords (Google, Lycos, etc.) • Easy to implement • Fast retrieval • Web image search (surrounding text) • Manual annotation is not always available • A picture is worth a thousand words • Surrounding text may not describe the image IEEE Int'l Symposium on Signal Processing and its Applications

  6. Content-Based Approach • Index images using low-level features CBIR System Image Database Content-based image retrieval (CBIR): search pictures as pictures IEEE Int'l Symposium on Signal Processing and its Applications

  7. A Data-Flow Diagram Histogram, color layout, sub-images, regions, etc. Image Database Feature Extraction Linear ordering, Projection to 2-D, etc. Euclidean distance, intersection, shape comparison, region matching, etc. Visualization Compute Similarity Measure IEEE Int'l Symposium on Signal Processing and its Applications

  8. Open Problem • Nature of digital images: arrays of numbers • Descriptions of images: high-level concepts • Sunset, mountain, dogs, …… • Semantic gap • Discrepancy between low-level features and high-level concepts • High feature similarity may not always correspond to semantic similarity IEEE Int'l Symposium on Signal Processing and its Applications

  9. Outline • Introduction • Cluster-based retrieval of images • Experiments • Conclusions and future work IEEE Int'l Symposium on Signal Processing and its Applications

  10. Motivation • A query image and its top 29 matches returned by a CBIR system Horses (11 out of 29), flowers (7 out of 29), golf player (4 out of 29) IEEE Int'l Symposium on Signal Processing and its Applications

  11. CLUE: CLUsters-based rEtrieval of images by unsupervised learning • Hypothesis In the “vicinity” of a query image, images tend to be semantically clustered • CLUE attempts to capture high-level semantic concepts by learning the way that images of the same semantics are similar IEEE Int'l Symposium on Signal Processing and its Applications

  12. System Overview A general diagram of a CBIR system using CLUE CLUE Image Database Select Neighboring Images Image Clustering Feature Extraction Display And Feedback Compute Similarity Measure IEEE Int'l Symposium on Signal Processing and its Applications

  13. Neighboring Images Selection • Nearest neighbors method • Pick k nearest neighbors of the query as seeds • Find r nearest neighbors for each seed • Take all distinct images as neighboring images k=3, r=4 IEEE Int'l Symposium on Signal Processing and its Applications

  14. Weighted Graph Representation • Graph representation • Vertices denote images • Edges are formed between vertices • Nonnegative weight of an edge indicates the similarity between two vertices IEEE Int'l Symposium on Signal Processing and its Applications

  15. Clustering • Graph partitioning and cut • Normalized cut (Ncut) [Shi et al., IEEE Trans. PAMI 22(8)] • Recursive Ncut IEEE Int'l Symposium on Signal Processing and its Applications

  16. Outline • Introduction • Cluster-based retrieval of images • Experiments • Conclusions and future work IEEE Int'l Symposium on Signal Processing and its Applications

  17. An Experimental System • Similarity measure • UFM[Chen et al. IEEE PAMI 24(9)] • Database • COREL • 60,000 IEEE Int'l Symposium on Signal Processing and its Applications

  18. User Interface IEEE Int'l Symposium on Signal Processing and its Applications

  19. Query Examples • Query Examples from 60,000-image COREL Database Bird, car, food, historical buildings, and soccer game UFM CLUE Bird, 6 out of 11 Bird, 3 out of 11 IEEE Int'l Symposium on Signal Processing and its Applications

  20. Query Examples CLUE UFM Car, 8 out of 11 Car, 4 out of 11 Food, 8 out of 11 Food, 4 out of 11 IEEE Int'l Symposium on Signal Processing and its Applications

  21. Query Examples CLUE UFM Historical buildings, 10 out of 11 Historical buildings, 8 out of 11 Soccer game, 10 out of 11 Soccer game, 4 out of 11 IEEE Int'l Symposium on Signal Processing and its Applications

  22. Clustering WWW Images • Google Image Search • Keywords: tiger, Beijing • Top 200 returns • 4 largest clusters • Top 18 images within each cluster IEEE Int'l Symposium on Signal Processing and its Applications

  23. Clustering WWW Images Tiger Cluster 1 (75 images) Tiger Cluster 2 (64 images) Tiger Cluster 3 (32 images) Tiger Cluster 4 (24 images) IEEE Int'l Symposium on Signal Processing and its Applications

  24. Clustering WWW Images Beijing Cluster 1 (61 images) Beijing Cluster 2 (59 images) Beijing Cluster 3 (43 images) Beijing Cluster 4 (31 images) IEEE Int'l Symposium on Signal Processing and its Applications

  25. Retrieval Accuracy IEEE Int'l Symposium on Signal Processing and its Applications

  26. Outline • Introduction • Cluster-based retrieval of images • Experiments • Conclusions and future work IEEE Int'l Symposium on Signal Processing and its Applications

  27. Conclusions • Retrieving image clusters by unsupervised learning • Tested using 60,000 images from COREL and images from WWW IEEE Int'l Symposium on Signal Processing and its Applications

  28. Future Work • Recursive Ncut • Representative image • Other graph theoretic clustering techniques • Nonlinear dimensionality reduction IEEE Int'l Symposium on Signal Processing and its Applications

  29. Thank You! IEEE Int'l Symposium on Signal Processing and its Applications

More Related