1 / 20

Leveraging Relative Visual Attributes for Enhanced Image Description

This publication discusses the usage of relative visual attributes for more detailed and informative image descriptions. By describing an image's attributes in relation to others, it allows for richer human supervision and potentially higher recognition accuracy. The proposed algorithm involves training a ranking function per attribute and predicting relative strengths on novel images, enabling tasks like zero-shot learning and generative models for joint ranking outputs. Experimental results show the effectiveness of this approach in face and outdoor scene recognition.

phoebe-patel
Télécharger la présentation

Leveraging Relative Visual Attributes for Enhanced Image Description

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Relative Attributes Speaker DengLei At I-VisionGroup

  2. Devi Parikh & Kristen Grauman • ICCV 2011 Marr Prize

  3. Publication—Devi Parikh

  4. … last 3 years: • ICCV 3(one only her) ECCV 1 CVPR 9 IJCV 1 NIPS 1

  5. Outline • Introduction • Algorithms • Experiments

  6. Introduction—Backgrounds • Visual attributes • Benefit various recognition tasks • Restrict on categorical label • Binaries are unnatural • Motivation • How to describe middle image • Relative description —  one image’s attribute strength with respect to others • E.g. less natural than left, more nature than right • Richer mode of communication • Allow more detailed human supervision (maybe higher recognition accuracy) • More informative descriptions of novels

  7. Proposal • Steps • Training – learn ranking function per attribute • Testing – predict the relative strength per attribute on novel image • New Tasks • Build generative model over joint space of ranking output • Zero-shot learning relates unseen to seen • E.g. 'bears are furrier than giraffes‘ • Enable richer textual description for new images • More precise • Tested on faces and natural scenes compared with binaries

  8. Outline • Introduction • Algorithms • Experiments

  9. Algorithms— learning relative attrs

  10. wide-margin {ranking VS binary} classifier

  11. Novel zero-shot learning • Setup • N total categories: S seen, U unseen (no images available) • S: described relative to each other via attrs (no need all pairs) • U: described relative to seen in (subset of ) attrs • Gaussian • Test by Max-likehood

  12. Auto gen relative textual desc of images • Img -> Img • Img -> Class • More info than bin

  13. Outline • Introduction • Algorithms • Experiments

  14. Experiments • Setup • Outdoor Scene Recognition (OSR) • I: 2668, C: 8, • Coast, forest, highway, inside-city, mountain, open-country, street, tall-building • Gist • Public Figures Face (PubFig) • I: 772, C: 8 • Alex, Clive, Hugh, Jared, Miley, Scarlett, Viggo, Zac • Concatenated gist and color feature

  15. Database — relative attrs • Marked By a colleague

  16. zero-shot learning

  17. Conclusion • Idea to learn relative visual attrs. • Two new tasks • Zero-shot learning • Img description • Based on relative description

  18. Thanks

More Related