1 / 16

Enhancing Slide Image Retrieval with Machine Learning Techniques

This preliminary study explores the potential of integrating image and presentation features with textual data to enhance slide image retrieval. By leveraging machine learning, specifically linear regression and classification techniques, we aim to improve the accuracy of image ranking in presentation slides. Our methods involve preprocessing, feature extraction, and relevance judgments from human participants. Initial results show promise, indicating that while textual features are critical, image and presentation-specific features also contribute significantly to the retrieval process. Future work includes refining these methods and expanding the dataset.

druce
Télécharger la présentation

Enhancing Slide Image Retrieval with Machine Learning Techniques

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Slide Image Retrieval: A Preliminary Study Guo Min Liew and Min-Yen Kan National University of Singapore Web IR / NLP Group (WING)

  2. Papers don’t store all of the information about a discovery Dataset Tools Implementation details / conditions They also don’t help a person learn the research Textbooks Slides Beyond Text in Digital Libraries We’ll focus on this JCDL: Beyond Text 2

  3. What is Slide Image Retrieval? • Hypothesis: image and presentation features can enhance text-based features JCDL: Beyond Text 3

  4. Training 3. Machine Learning Relevance Judgments Presentation Slides Model 1. Pre- Processing 2. Feature Extraction Text + Image + Presentation Features Testing 4. Image Ranking Queries Presentation Slides Slidir Architecture JCDL: Beyond Text 4

  5. 1. Preprocessing Simple preprocessing, not meant to be state-of-the-art Generally consistent background  Perform collective separation Top down approach using projection profile cuts with thresholds from PPT template JCDL: Beyond Text 5

  6. Sample Extraction Results JCDL: Beyond Text 6

  7. 2. Feature Extraction Utilizes textual, image and presentation features • Textual Features • Main component of feature set • Most images identified by surrounding and embedded text • Image Features • Supplement to textual features • Useful in ranking images with similar textual feature scores • Presentation Features • Features unique to presentation slides JCDL: Beyond Text 7

  8. Sample Features Image • Image Size • Image Colors • NPIC image type (CIVR, 06) Presentation • Slide Order • Image position on Slide • Number of Images on Slide Textual All text 2. Slide text 3. Next slide’s text 4. Previous slide’s text 5. Slide Title 6. Presentation Title 7. Slide Image Text … JCDL: Beyond Text 8

  9. Machine Learning Regression Model • Relevance judgments and feature vectors input into machine learner • Linear regression used to produce continuous output value for image ranking Human Relevance Judgments • 9 participants • Four presentation sets of 10 queries each • Participants rank up to five most relevant images to the query JCDL: Beyond Text 9

  10. Regression Analysis • 10-fold cross validation • System with fielded text features able to replicate human upper bound • Image and presentation features further improve correlation Still a gap toimprove Looks pretty good Comparison of mean absolute error and correlation JCDL: Beyond Text 10

  11. Binary Classification • Human participants judgments converted to binary • Evaluated using SVM and J48 decision tree classifier • General accuracy fairly good • Decision tree algorithm slightly more precise  sequential testing adequate 10-fold performance over 8062 instances JCDL: Beyond Text 11

  12. Feature Analysis • Human subject study with 20 volunteers • Subjects select most and least relevant images Sample Human Subject Survey JCDL: Beyond Text 12

  13. Textual features most important, image and presentation features still play significant role JCDL: Beyond Text 13

  14. Image Ranking Evaluation • Online survey with 15 volunteers • 8 given queries and sampled dataset of 100 images • Volunteers indicate relevant images to query • 53% of top 10 images are relevant (max of 15 relevant) • Average precision of 0.74 JCDL: Beyond Text 14

  15. Limitations Very small data set – 100 images from an AI course Preprocessing Improve segmentation and collective extraction Field Implementation – Ongoing work Incorporate with previous work on SlideSeer (JCDL, 2007) that aligns presentation and document pairs More information: Liew Guo Min(2008) SLIDIR: Slide Image Retrieval, Undergraduate Thesis, School of Computing, National University of Singapore. Future Work JCDL: Beyond Text 15

  16. See you in Singapore! Next month or next year! Questions? JCDL: Beyond Text

More Related