SUBJECT SPECIFIC FACE RECOGNITION

1. SUBJECT SPECIFIC FACE RECOGNITION J.W.H. Tangelder (CWI), M. Bicego (UNISS) D. González Jiménez (UVIGO) J.L. Alba-Castro (UVIGO), E. Grosso(UNISS) M. Tistarelli(UNISS), B.A.M. Schouten(CWI)

2. Overview • Face recognition using most dissimilar patches (UNISS) • Face recognition using most discriminative Gabor features (UVIGO) • Authentication by Bayesian fusion of subject specific face descriptors (CWI) • Future research directions

3. Face recognition using most dissimilar patches • Key concepts • Subject specific face recognition • Face distinctiveness • Subject specific face recognition: • A template should be tailored to the subject peculiarities • Face distinctiveness: • Use as template “the most distinctive parts” of the subject’s face

4. Computing most distinctive parts Distinctiveness is related to saliency But…. Saliency is an “unary” operation [Tsotsos et al. 95][Lindeberg 93][Salah et al. 02][Lowe 04] Distinctiveness is related to a n-ary operation: “Finding differences between the client face and all other faces”

5. Computing most distinctive parts Basic scheme • Binary operator: finding differences between a pair of faces • Face sampling, multiscale patch extraction and description • Projection in the feature spaces and weighting

6. Computing most distinctive parts • Face sampling, multiscale patch extraction and description: • Sampling points are the edges of the face + random points • Logpolar patches are extracted at each point

7. Log polar patches: • Multiresolution information • they describe the mapping postulated to occur between the retina and the visual cortex • [Grosso Tistarelli 2000]

8. Face 1 Face 2 y Most distinctive patches Confusion: most similar patches x Most distinctive patches • Projection in the feature space and weighting The weight of a patch is proportional to the distance to the other set

9. Generalization to many to many • Simple generalization: • Extract and project all client faces patches • Extract and project all impostor faces patches • Compute most distinctive client patches: • Patches which are very different from all other impostor patches (the rest of the world) are the most distinctive • Obviously it depends on the size of the impostor set

10. Basic experimental evaluation: an “impairment” test • Idea: given a simple authentication scheme (Euclidean distance between images) • Compare: • Method 1: delete from the images the K most important patches (of the claimed identity) • Method 2: delete the same amount of random points • Look for the “worst behaviour”

11. Methodology HTER (R=0.1) HTER (R=1) HTER (R=10) Using all points 17.31 17.35 17.39 Deleting random points 18.22 18.39 18.45 Deleting most distinctive points 23.04 22.88 22.72 Results • Banca database: protocol MC • Result: deleting the most distinctive points is really deleterious for autentication • Next step: design a “direct” face authentication methodology

12. Face recognition using most discriminative Gabor features • Gabor features selection is splitted into 2 stages: • First Layer: A set of points are selected by exploiting facial structure, and Gabor features are computed. • Second Layer: A subset of the initial group of points are preserved based on their respective Gabor features’ accuracy

13. First Layer: Shape-driven location • Ridges & Valleys detector • Sampling points from lines depicting facial structure • Preliminary set of points for client C :

14. Extracting texture • 40 Gabor filters are used • A feature vector (Gabor jet) is extracted at each shape-driven point • Set of jets: Jpk

15. Second Layer: Accuracy-based selection (I) • Goal: Given a set of: • Available images for client C • Available impostor images Find the best subset of nodes for client C: ..... .....

16. Second Layer: Accuracy-based selection (II) • Each jet is evaluated as an individual classifier • Only the nodes whose jets are good at discriminating between client C and impostors are preserved. • Finally, for client C, we get:

17. Results • Banca database: protocol P. ERROR RATES (%)

18. Authentication by Bayesian Fusion of Subject Specific Face Descriptors • Idea: • Extract features at most discriminative face parts for both clients and impostors • Model the distribution of features by both a Gaussian client model and a Gaussian impostor model • Take an authentication decision by thresholding the sum of log-likelihood-ratios of the client and impostor

19. Client and impostor feature templates Template of client feature models built on 5 locations using 5 client images Template of impostor feature modes built on 5 locations using 5 world images

20. Gaussian client and impostor feature models For each client c, for each location we estimate the probability of feature value x by a Gaussian client model of its atypical client-specific feature distribution And by a Gaussian non-client model of its typical client-specific feature distribution

21. Taking the authentication decision For each location the log-likelihood ratio is used to assign confidence that a feature value is client-specific: Based on the weighted sum of these log-likelihood-ratios over all reference points the claim of a client c is accepted when it exceeds a given threshold reject accept

22. Future research directions • Generalizing methods developed for authentication to face recognition • Fusion using both Gabor jets and log polar patch features • Developing new approaches to compute the matching from the features in one image to the features in another image • Applying a cascaded approach to face recognition: Compare first, the most salient part of a face against other faces • Applying methods to video (tracking landmarks, pose correction)