A Literature Review of Image-based Face Recognition

1. A Literature Review of Image-based Face Recognition Quan Ju PhD student Department of Computer Science The University of York The University of York

2. Background Introduction • What is Face Recognition? • Face Recognition is a popular application of computer vision in recent years. • Not only the computer science researchers, but also the psychologists and neuroscientists are involved in this area. • Why do we need Face Recognition? • Strong need for personal identification and recognition without the cooperation of the participants. • Commercial, security and law applications require the use of face recognition technology. The University of York

3. Face Recognition Scenarios • Face Verification: Is this person who he says he is? One to one matching process. • Face Identification: Who is this person? One to many matching process • Watch list: Who are you looking for? The University of York

4. Difficulties in Face Recognition • Head pose • Illumination • Facial expression • Hair • Aging problem • Occlusion e.g. glasses, scarf etc. The University of York

5. Image-based Face recognition approaches • Appearance-based face recognition Linear Analysis: PCA,ICA,LDA Non-linear analysis • Model-based face recognition Elastic Bunch Graph Matching 2D Morphable Model 3D Morphable Model The University of York

6. Linear Analysis • Classical linear appearance-based analysis - PCA, ICA and LDA each has its own basis vectors of a high dimensional face image space. • By using those linear analysis method, the face vectors can be projected to the basis vectors. • Through the projecting from a higher dimensional input image space to a lower dimensional space, dimensionality of original input image space is reduced. • The matching score between the test face image and training images can be achieved by calculation the differences between their projection vectors. The higher the score, the more similar between these two face images. The University of York

7. Image Space • Image vector and image subspace x1 represents a p×q image; x is a matrix of image vectors. Above is three 1x2 pixel image examples. Similar images locate close together, otherwise they are away from each other. The University of York

8. Principal Component Analysis • The main idea of the principal component analysis is to find the vectors which best describe the distribution of face images within the entire image space. • PCA is an orthogonal transformation of the coordinate system in which the pixels are described. The new coordinate values are principal component • Face space is comprised of eigenfaces, which are the eigenvectors of the set of the faces. • PCA is performed by projecting a new image into the subspace called face space spanned by the eigenfaces and then classifying the face by comparing its position in face space with the positions of known individuals. • PCA aims to extract a subspace where the variance is maximized The University of York

9. Independent Component Analysis • PCA derives only the most expressive features which are unrelated to actual face recognition, and in order to improve performance additional discriminant analysis is needed. • ICA provide a more powerful data representation than PCA as its aim is to provide an independent rather than uncorrelated image decomposition and representation. • ICA is a generalization of PCA. The University of York

10. Linear Discriminant Analysis • Similar images projections are close together, different images projections locate far away when using PCA, but the projection from different classes of images are mixed together. • LDA is also called Fisher Discriminant Analysis • LDA is able to maximize the ratio of between-class distribution to that of within-class distribution. The University of York

11. Nonlinear analysis • Linear discriminant methods are insensitive to the relationship among multiple pixels in the images. Some nonlinear relations may exist in a face image, especially under a complicated variation in viewpoint, illumination and facial expression which is highly nonlinear. • To extract nonlinear features of images, Linear analysis method was extended to nonlinear analysis such as Kernel PCA, Kernel ICA and Kernel FLD etc. • By using nonlinear analysis approaches the original input image space is projected nonlinearly onto a high dimensional feature space. In this high dimensional space, the distribution of image vectors could be simplified to linear pattern. The University of York

12. Model-based face recognition • The model-based face recognition scheme is aimed at constructing a model of the human face, which is able to capture the facial variations. • Model-based approaches derive distance and relative position features from the placement of internal facial elements (eyes, nose). • Generally, a face model contains the information of shape and texture of the face. The University of York

13. Bunch Graph • Human faces have a similar topological structure. • Face can be structured by nodes located at some specific points and edges labeled with distance vectors, then a face graph is produced. • Face Bunch graph is generated from a set of sample face images. The FBG serves as a general representation of a set of faces. • The stacks of discs on a node contain a bunch of description of facial features. • Each stack of discs called a jet represents an alternative of facial feature description. • The edges are labeled with averages of distance vectors. The University of York

14. Elastic Bunch Graph Matching • To recognize a new face by elastic bunch graph matching, the fiducial points are positioned so as to extract a graph, which maximize a graph similarity between this graph and the FBG. • After the nodes has been located on the new face, the face can be recognized by comparing the similarity between that the graph of this face and graphs of every face store in the FBG. The University of York

15. An Active Appearance Model • The AAM is constructed based on a set of labeled images, where landmark points are marked on each example face at key positions to describe the facial features. • Models are combined together by using Linear Analysis methods such as PCA. • The vector of parameters for the combined model is controlling the shape and texture of models. • AAM fitting is applied to seek a set of model parameters that best represents the test face image. • The goal of recognition is to find the best match between the test parameter vector and training parameter vector. The University of York

16. 3D Morphable model • Human face is a surface lying in the 3D space. Thus, the 3D model is more suitable for representing faces, • 3D model has stronger ability to minimize the problems of head pose, illumination. • 3D morphable model is extended from 2D morphable model - AAM. • Similar recognition methods on 2D morphable model can be improved and applied on 3D model as well. The University of York

17. Face Databases and Performance Evaluation • There are about 28 face databases available currently, such as FERET, XM2VTS and UMIST etc. • How to choose the suitable database based on the task given and the algorithm needs. • FERET is a poplar face image database, which contains 1564 sets of images for a total of 14,126 images that includes 1199 individuals and 365 duplicate sets of images. • False Acceptance / False Rejection and Equal Error Rate are scores to evaluate the similarity between a test pattern and a template. • The Face Recognition Vendor Test was started from 2000 based on the FERET database. The database used in FRVT was extended 2 years later in FRVT2002. The University of York

18. References • W. Zhao, R. Chellappa, A. Rosenfeld, P.J. Phillips, Face Recognition: A Literature Survey, ACM Computing Surveys, 2003, pp. 399-458 • X. Lu, Image Analysis for Face Recognition, personal notes, May 2003, 36 pages • H. Moon, P.J. Phillips, Computational and Performance aspects of PCA-based Face Recognition Algorithms, Perception, Vol. 30, 2001, pp. 303-321M.A. Turk, A.P. Pentland, Face Recognition Using Eigenfaces, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3-6 June 1991, Maui, Hawaii, USA, pp. 586-591 • M.A. Turk, A.P. Pentland, Face Recognition Using Eigenfaces, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3-6 June 1991, Maui, Hawaii, USA, pp. 586-591 • C. Liu, H. Wechsler, Comparative Assessment of Independent Component Analysis (ICA) for Face Recognition, Proc. of the Second International Conference on Audio- and Video-based Biometric Person Authentication, AVBPA'99, 22-24 March 1999, Washington D.C., USA, pp. 211-216 • A. Pentland, B. Moghaddam, T. Starner, View-Based and Modular Eigenspaces for Face Recognition, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 21-23 June 1994, Seattle, Washington, USA, pp. 84-91 • K. Etemad, R. Chellappa, Discriminant Analysis for Recognition of Human Face Images, Journal of the Optical Society of America A, Vol. 14, No. 8, August 1997, pp. 1724-1733 • W. Zhao, R. Chellappa, A. Krishnaswamy, Discriminant Analysis of Principal Components for Face Recognition, Proc. of the 3rd IEEE International Conference on Face and Gesture Recognition, FG'98, 14-16 April 1998, Nara, Japan, p. 336 • A.M. Martinez, A.C. Kak, PCA versus LDA, IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 23, No. 2, 2001, pp. 228-233 • J. Lu, K.N. Plataniotis, A.N. Venetsanopoulos, Face Recognition Using LDA-Based Algorithms, IEEE Trans. on Neural Networks, Vol. 14, No. 1, January 2003, pp. 195-200 • C. Liu, H. Wechsler, Evolutionary Pursuit and Its Application to Face Recognition, IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 22, No. 6, June 2000, pp. 570-582 • L. Wiskott, J.-M. Fellous, N. Krueuger, C. von der Malsburg, Face Recognition by Elastic Bunch Graph Matching, Chapter 11 in Intelligent Biometric Techniques in Fingerprint and Face Recognition, eds. L.C. Jain et al., CRC Press, 1999, pp. 355-396 • M.-H. Yang, Kernel Eigenfaces vs. Kernel Fisherfaces: Face Recognition Using Kernel Methods, Proc. of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition, 20-21 May 2002, Washington D.C., USA, pp. 215-220 • .-H. Yang, Face Recognition Using Kernel Methods, Advances in Neural Information Processing Systems, T. Diederich, S. Becker, Z. Ghahramani, Eds., 2002, vol. 14, 8 pages • T.F. Cootes, K. Walker, C.J. Taylor, View-Based Active Appearance Models, Proc. of the IEEE International Conference on Automatic Face and Gesture Recognition, 26-30 March 2000, Grenoble, France, pp. 227-232 • V. Blanz, T. Vetter, Face Recognition Based on Fitting a 3D Morphable Model, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 25, No. 9, September 2003, pp. 1063-1074 • B. Moghaddam, J.H. Lee, H. Pfister, R. Machiraju, Model-Based 3D Face Capture with Shape-from-Silhouettes, Proc. of the IEEE International Workshop on Analysis and Modeling of Faces and Gestures, AMFG, 17 October 2003, Nice, France, pp. 20-27 • J. Lee, B. Moghaddam, H. Pfister, R. Machiraju, Finding Optimal Views for 3D Face Shape Modeling, Proc. of the International Conference on Automatic Face and Gesture Recognition, FGR2004, 17-19 May 2004, Seoul, Korea, pp. 31-36 The University of York