1 / 18

Finding Celebrities in Billions of Web Images

Finding Celebrities in Billions of Web Images. 云飞 2012-12-13. Overview. 一、 label an input image with a list of celebrities. 二、 the celebrity names are assigned to the faces by label propagation on a facial similarity graph. Overview. 本文的优点：

lukas
Télécharger la présentation

Finding Celebrities in Billions of Web Images

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Finding Celebrities in Billions of Web Images 云飞 2012-12-13

  2. Overview • 一、label an input image with a list of celebrities. • 二、the celebrity names are assigned to the faces by label propagation on a facial similarity graph.

  3. Overview • 本文的优点： • 1、the proposed image annotation system is capable of labeling names to general web images. • 2、our name assignment algorithm does not impose any assumption on the facial feature distribution. • 3、not only visual cues are used.

  4. Overview • 1. determine, by identifying celebrity names from surrounding text. • 2. given a set of names, assign the names to the faces in the input image.

  5. Overview • A. Image Annotation System • 1) construct a vocabulary; • 2) discover all webpages hosting its near-duplicates; • 3) use the vocabulary to filter the surrounding text. • Advances： • 1)effective; • 2)remove noise. • Annotated images: • 1)SFSN • 2)SFMN • 3)MF

  6. Overview • B. Multimodal Name Assignment • The context likelihood incorporates the information from surrounding text by using the confidence scores estimated by the image annotation system.

  7. IMAGE ANNOTATION SYSTEM • Goal: label an input image with a list of celebrities who may appear in the image. • A. Constructing a Large-Scale Celebrity Name Vocabulary • B. Discover Related Webpages by Near-Duplicate Image Retrieval • C. Annotating Images by Mining Surrounding Text of Related Webpages

  8. IMAGE ANNOTATION SYSTEM • Constructing a Large-Scale Celebrity Name Vocabulary 1)Wikipedia 首段 信息框 标签 2)Entitycube

  9. IMAGE ANNOTATION SYSTEM • B. Discover Related Webpages by Near-Duplicate Image Retrieval • divide and conquer strategy • 图片分成n×n • 降维 • 阈值化

  10. IMAGE ANNOTATION SYSTEM • C. Annotating Images by Mining Surrounding Text of Related Webpages • 1) Type of names； • 2) Type of surrounding text； • 3) Frequency versus ratio；

  11. MULTIMODAL NAME ASSIGNMENT • A. Notation • B. Overview of the Assignment Model • C. Label Propagation from SFSN Images p(Y|F) • D. Constrain the Propagation by a Context Likelihood p(Y|T; λ) • E. Normalization by Name Prior p(Y) • F. Implementation Detail: Face Representation

  12. A. Notation • faces in image In • denote the face labels as

  13. B. Overview of the Assignment Model • the confidence for label

  14. C. Label Propagation from SFSN Images p(Y|F) • how to propagate labels from SFSN images to SFMN and MF images

  15. D. Constrain the Propagation by a Context Likelihood p(Y|T; λ) • 1) For each image-level name vk, generate a binary variable zk from p(vk |T) as defined in (3) to indicate whether vk appears in image I. • 2) If zk=1, generate mk faces of name vk in image I from p(m|z; λ) as defined in (13).

  16. E. Normalization by Name Prior p(Y) • p(Y) represents the prior of names.

  17. F. Implementation Detail: Face Representation • the appearance of each face is described by local binary pattern (LBP). • the face image is divided into small regions from which the LBP features are extracted and concatenated into a single feature histogram. • pply PCA to reduce the dimension of face descriptor from over 3000 to 500 dimensions.

  18. Evaluation

More Related