Face Recognition

1. Face Recognition A Literature Review By Xiaozhen Niu Department of Computing Science

2. Contents • Face Segmentation/Detection • Facial Feature extraction • Face Recognition • Video-based Face Recognition • Comparison • Summary • Reference

3. Face Segmentation/Detection Before the middle 90’s, the research attention was only focused on single-face segmentation. The approaches included: • Deformable feature-based template • Neural network • Using skin color

4. Face Segmentation/Detection During the past ten years, considerable progress has been made in multi-face recognition area, includes: • Example-based learning approach by Sung and Poggio (1994). • The neural network approach by Rowley et al. (1998). • Support vector machine (SVM) by Osuna et al. (1997).

5. Example-based learning approach (EBL) Three parts: • The image is divided into many possible-overlapping windows, each window pattern gets classified as either “a face” or “not a face” based on a set of local image measurements. • For each new pattern to be classified, the system computes a set of different measurements between the new pattern and the canonical face model. • A trained classifier identifies the new pattern as “a face” or “not a face”.

6. Example of a system using EBL

7. Neural network (NN) • Kanade et al. first proposed an NN-based approach in 1996. • Although NN have received significant attention in many research areas, few applications were successful. Why?

8. Neural network (NN) • It’s easy to train a neural network with samples which contain faces, but it is much harder to train a neural network with samples which do not. • The number of “non-face” simples are just too large.

9. Neural network (NN) • Neural network-based filter. A small filter window is used to scan through all portions of the image, and to detect whether a face exists in each window. • Merging overlapping detections and arbitration. By setting a small threshold, many false detections can be eliminated.

10. An example of using NN

11. Test results of using NN

12. SVM • SVM was first proposed in 1997, it can be viewed as a way to train polynomial neural network or radial basic function classifiers. • Can improve the accuracy and reduce the computation.

13. Comparison with EBL • Test results reported in 1997. • Using two test sets (155 faces). SVM achieved better detection rate and fewer false alarms.

14. Recent approaches Face segmentation/detection area still remain active, for example: • An integrated SVM approach to multi-face detection and recognition was proposed in 2000. • A technique of background learning was proposed in August 2002. Still lots of potential!

15. Static face recognition Numerous face recognition methods/algorithms have been proposed in last 20 years, several representative approaches are: • Eigenface • LDA/FDA • Neural network (NN)

16. Eigenface The basic steps are: • Registration. A face in an input image first must be located and registered in a standard-size frame. • Eigenpresentation. Every face in the database can be represented as a vector of weights, the principal component analysis (PCA) is used to encode face images and capture face features. • Identification. This part is done by locating the images in the database whose weights are the closest (in Euclidean distance) to the weights of the test images.

17. LDA/FDA • Face recognition method using LDA/FDA is called the fishface method. • Eigenface use linear PCA. It is not optimal to discrimination for one face class from others. • Fishface method seeks to find a linear transformation to maximize the between-class scatter and minimize the within-class scatter. • Test results demonstrated LDA/FDA is better than eigenface using linear PCA (1997).

18. Test results of LDA • Test results of a subspace LDA-based face recognition method in 1999.

19. Video-based Face Recognition • Three challenges: • Low quality • Small images • Characteristics of face/human objects. • Three advantage: • Allows Provide much more information. • Tracking of face image. • Provides continuity, this allows reuse of classification information from high-quality images in processing low-quality images from a video sequence.

20. Basic steps for video-based face recognition • Object segmentation/detection. • Motion structure. The goal of this step is to estimate the 3D depths of points from the image sequence. • 3D models for faces. Using a 3D model to match frontal views of the face. • Non-rigid motion analysis.

21. Recent approaches Most video-based face recognition system has three modules for detection, tracking and recognition. • An access control system using Radial Basis Function (RBS) network was proposed in 1997. • A generic approach based on posterior estimation using sequential Monte Carlo methods was proposed in 2000. • A scheme based on streaming face recognition (SFR) was propose in August 2002.

22. The SFR scheme • Combine several decision rules together, such as Discrete Hidden Markov Models (DHMM) and Continuous Density HMM (CDHMM). The test result achieved a 99% correct recognition rate in the intelligent room.

23. Comparison Two most representative and important protocols for face recognition evaluations: • The FERET protocol (1994). • Consists of 14,126 images of 1199 individuals. • Three evaluation tests had been administered in 1994, 1996, and 1997. • The XM2VTS protocol (1999). • Expansion of previous M2VTS program (5 shots of each of 37 subjects). • Now consists 295 subjects. • The results of M2VTS/XM2VTS can be used in wide range of applications.

24. 1996/1997 FERET Evaluations • Compared ten algorithms.

25. Summary • Significant achievements have been made. LDA-based methods and NN-based methods are very successful. • FERET and XM2VTS have had a significant impact to the developing of face recognition algorithms. • Challenges still exist, such as pose changing and illumination changing. Face recognition area will remain active for a long time.

26. Reference [1] W. Zhao, R. Chellappa, A. Rosenfeld, and P.J. Phillips, Face Recognition: A Literature Survey, UMD CFAR Technical Report CAR-TR-948, 2000. [2] K. Sung and T. Poggio, Example-based Learning for View-based Human Face Detection, A.I. Memo 1521, MIT A.I. Laboratory, 1994. [3] H.A. Rowley, S. Baluja, and T. Kanade, Neural Network Based Face Detection, IEEE Trans. On Pattern Analysis and Machine Intelligence, Vol. 20, 1998. [4] E. Osuna, R. Freund, and F. Girosi, Training Support Vector Machines: An Application to Face Recognition, in IEEE Conference on Computer Vision and Pattern Recognition, pp. 130-136, 1997. [5] M. Turk and A. Pentland, Eigenfaces for Recognition, Journal of Cognitive Neuroscience, Vol.3, pp. 72-86, 1991. [6] W. Zhao, Robust Image Based 3D Face Recognition, PhD thesis, University of Maryland, 1999. [7] K.S. Huang and M.M. Trivedi, Streaming Face Recognition using Multicamera Video Arrays, 16th International Conference on Pattern Recognition (ICPR). August 11-15, 2002. [8] P.J. Phillips, P. Rauss, and S. Der, FERET (Face Recognition Technology) Recognition Algorithm Development and Test Report, Technical Report ARL-TR 995, U.S. Army Research Laboratory. [9] K. Messer, J. Matas, J. Kittler, J. Luettin, and G. Maitre, XM2VTSDB: The Extended M2VTS Database, in Proceedings, International Conference on Audio and Video-based Person Authentication, pp. 72-77, 1999.

27. Questions