Unsupervised Face Annotation by Mining the Web

1. Unsupervised Face Annotation by Mining the Web Data Mining, 2008. ICDM '08. Eighth IEEE International Conference on Duy-Dinh Le National Institute of Informatics 2-1-2 Hitotsubashi, Chiyoda-ku Tokyo, JAPAN 101-8430 ledduy@nii.ac.jp Shin’ichi Satoh National Institute of Informatics 2-1-2 Hitotsubashi, Chiyoda-ku Tokyo, JAPAN 101-8430 satoh@nii.ac.jp Student: Tu , Chien-Hsun 69821059 LIU, Yuan-Ming 69821039

2. Outline • Introduction • Proposed Framework -Face Processing -Ranking by Local Density Score -Ranking by Bagging of SVM Classifier • Experimental Results • Conclusion

3. Introduction • Large image and video databases have become more available than ever to users. • This trend has shown the need for effective and efficient tools for indexing and retrieving based on visual content.

4. Introduction • Improve the retrieval performance is to take into account visual information present in the retrieved faces. • Challenge - Facial appearance due to pose changes, illumination ,facial expressions make face recognition difficult. - No labels makes supervised and unsupervised learning methods inapplicable.

5. System Framework

6. Proposed Framework- Face Processing • We perform a ranking process and learning of person X’s model as follows: • Step 1: Detect faces and eye positions, and then perform face normalizations.

7. Proposed Framework • Step 2: Compute an eigenface space and project the input faces into this subspace. • Step 3: Estimate the ranked list of these faces using Rank-By-Local-Density-Score. • Step 4: Improve this ranked list using Rank-By-Bagging-ProbSVM.

8. Ranking by Local Density Score • Among the faces retrieved by text-based search engines for a query of person-X, relevant faces usually look similar and form the largest cluster.

9. Ranking by Local Density Score • One approach of re-ranking these faces is to cluster based on visual similarity. • Problem Ideal clustering results is impossible since these faces are high dimensional data and the clusters are in different shapes, sizes, and densities.

10. Ranking by Local Density Score • We use the idea of density-based clustering described by to solve this problem. • We define the local density score (LDS) of a point p (i.e. a face) as the average distance to its k-nearest neighbors.

11. Ranking by Local Density Score • We do not directly use the Euclidean distance between two points in this feature space for distance(p, q).

12. Ranking by Local Density Score • A high value of LDS(p, k) indicates a strong association between p and its neighbors. Therefore, we can use this local density score to rank faces.

13. Ranking by Bagging of SVM Classifiers • Problem • One limitation of the local density score based ranking is it cannot handle faces of another person strongly associated in the k-neighbor set (for example, many duplicates). • The main idea is to use a probabilistic model to measure the relevancy of a face to person-X, P(person−X|face).

14. Ranking by Bagging of SVM Classifiers • Improving an input rank list by combining weak classifiers trained from subsets annotated by that rank list. We set p=20% : the maximum Kendall tau distance. (set=0.05)

15. Ranking by Bagging of SVM Classifiers • The iterations significantly improve the final ranked list.

16. Experimental Results

17. Experimental Results • We performed a comparison between our proposed method with other existing approaches. • Text Based Baseline (TBL) • Distance-Based Outlier (DBO) • Densest Sub-Graph based Method (DSG) • Local Density Score (LDS) • Unsupervised Ensemble Learning Using Local Density Score (UEL-LDS) • Supervised Learning (SVM-SUP)

18. Experimental Results Performance comparison of methods.

19. Experimental Results Distribution of retrieved faces and relevant faces of 16 individuals used in experiments.

20. Experimental Results-Evaluation Criteria • Nret be the total number of faces returned, • Nrel the number of relevant faces • Nhit the total number of relevant faces

21. Experimental Results

22. Conclusion • Our approach works fairly well for well known people, where the main assumption that text-based search engines return a large fraction of relevant images is satisfied. • The aim of our future work is to study how to improve the quality of the training sets used in this iteration (bagging SVM classifiers).