1 / 22

Similarity-based matching for face authentication

Similarity-based matching for face authentication. Christophe Rosenberger Luc Brun ICPR 2008. outline. Introduction Image description Dissimilarity measure Pattern modeling Experimental results. Introduction.

istas
Télécharger la présentation

Similarity-based matching for face authentication

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Similarity-based matching for face authentication Christophe Rosenberger Luc Brun ICPR 2008

  2. outline • Introduction • Image description • Dissimilarity measure • Pattern modeling • Experimental results

  3. Introduction • Propose in this paper a face authentication method based on a similarity measure • Find interest key points • SIFT descriptor is used • A graph is then built where nodes correspond to these keypoints • Model the authentication problem as a graph matching process

  4. Image description • SIFT (Scale Invariant Feature Transform) • Applied locally on key-points • Based upon the image gradients in a local neighborhood • SIFT detector • Scale-space extrema detection • Keypoint localization • SIFT descriptor • Orientation assignment • Keypoint descriptor

  5. SIFT detector • Detection of scale-space extrema • DoG(Difference-of-Gaussian) filter for scale space • Keypoint localization (local extrema) X is selected if it is larger or smaller than all 26 neighbors

  6. SIFT detector & descriptor • Accurate keypoint localization • Reject points with low contrast (flat) • Elimination edge responses • Orientation assignment • To make the feature invariant to rotation • Find local major orientation • Orientation histogram for a window size • Local image descriptor • 16x16 array of locations in scale space • Create array of orientation histograms • 8 orientations x 4x4 histogram array (128D) Gradient magnitude Gradient orientation

  7. Dissimilarity measure • Each individual is described by a set of invariant features Y (I) = {ki = (si, xi, yi)}, i = 1 : N(I) • Si is the SIFT invariant descriptor • (xi, yi) its position in the original image I • N(I) the number of detected keypoints in the image I • Database image • {Y (Ii), i = 1 : M} • Dissimilarity between two sets of points such as Y (I1) and Y (I2) • Often modeled as a weighted bipartite graph matching problem • Greedy algorithm • Hungarian method • Hybrid method

  8. Greedy algorithm • Weighted bipartite graph G(U,V,E,W) • Given the graph G , scan the set of edges and find an edge (u,v) with the smallest weight • Match u with v and remove the edges incident on u and those incident on v • Repeat the above step until every node in U is matched to some node in V 3 3 10 12 5 5 20 25 6 4 9 4

  9. Greedy algorithm • The greedy algorithm does not always produce a matching of minimum weight • Hungarian method • A well-known algorithm for finding a minimum weight matching in a bipartite graph • Node in U by u1, u2,… un • In V by v1, v2,… vn • x ij= 1 if ui is matched with vjand 0 otherwise subject to constraints 1 A A C C 16 16 0 0 B 2 D B D Each node in U is matched to exactly one node in V and vice versa

  10. Hungarian method - Example Step 1 First for each row we subtract the row minimum from the rest of the row R1 B1 R2 B2 R3 B3 R4 B4 Step 2 Then for each column we subtract the column minimum from the rest of the column

  11. Hungarian method - Example R1 B1 Step 4 1.Find the min value not covered (B3-R1) 2.subtracted from each uncovered entry and added to each entry covered by both a vertical and horizontal line (B1-R2 & B1-R4) 3. Step 3 is then repeated Step 3 Cover the min number of rows or columns required to cover all of the zeros in the matrix R2 B2 R3 B3 R4 B4 If the # of line = the # of rows/columns in the matrix then we are finished Otherwise, we have to go to the next step 3 Result: Blue1 - Red3Blue2 - Red2Blue3 - Red1Blue4 – Red4 4 13 2

  12. Hybrid method • We formulated the point matching problem in terms of minimum weight matching in a bipartite graph • In a real world situation, this is not always appropriate • Feature points extracted from two corresponding images • First image are denoted by a and b • Second image are denoted by c and d One might argue that since b and c both have the same feature value of 100,we should match them together and completely ignore a and d

  13. Hybrid method • Feature points forming tight clusters in the feature space can be avoided by a simple combination of the greedy method and the Hungarian method • Combined algorithm • We first apply the Hungarian method to get a matching M1 • Apply the greedy algorithm to get a matching M2 • form their intersection M=M1∩M2 Greedy algorithm Combined matching A bipartite graph Hungarian method

  14. Dissimilarity measure • Such a weighted bipartite graph matching method • Optimizes a global matching score • May thus be sensible to outliers • We thus rather use the following matching method • Given two points and , we say that x is associated to y iff : • C is an arbitrary threshold • d(., .) denotes the Euclidean distance between SIFT descriptors • y’ denotes any point of Y (I2) whose distance to x is minimal but greater than d(x,y)

  15. Dissimilarity measure • The significance of the necessary gap between d(x, y) and d(x, y’) is encoded by the constant C • y is associated to x • x is the closest point from y among Y (I1) • second smallest value of this distance d(y, x’) satisfies d(y, x) < Cd(y, x’) • X is matched to Y • iff x is associated to y and y is associated to x • If Y (I1) and Y (I2) do not define any matching point • Both sets have an infinite distance • Otherwise • The set of matching points M1 of Y1 and M2 of Y2 may be computed sequentially • If we cannot find two couples of points within Y (I1) × Y (I2) with a similar distance, the relationship “is matched to “ induces a bijective mapping Φ from M1 to M2

  16. Dissimilarity measure • First step • The above procedure classify both sets Y (I1) and Y (I2) • Dissimilar (case M1 = M2 =ø) • A set of N = |M1| = |M2| couples (x, Φ(x)) of matched points • Second step • Take into account the spatial relationship of the points among M1 & M2 • Use one of the four pattern representations • Using the locations of each matched keypoint • Weighted graph • Edges are weighted by the similarity of the adjacent points • The matching between M1 and M2being encoded by the application Φ • Encode the adjacency matrices of both graphs • A line or a column encoding the adjacency relationships • Point x of the first matrix • Φ(x) in the second matrix Graph Adjacency matrix

  17. Pattern modeling • Spatial relationship between keypoints • Using graph representations • Connect keypoints by considering their ”local neighborhoods” • Each keypoint corresponds to a vertex in the neighborhood graph which connects “close” vertices • Epsilon-neighborhood graph (ENG) • Two vertices are connected if the distance of the corresponding keypoints is less than epsilon • A completely connected graph (CCG) • epsilon greater than the maximal distance between keypoints.

  18. Pattern modeling • Symmetric k-nearest neighbor graph (SKNG) • Two vertices V1, V2are connected if V1 is among the k-nearest neighbors of V2 or vice versa • Mutual k-nearest neighbor graph (MKNG) • Two vertices V1, V2 are connected if V1 is among the k-nearest neighbors of V2 and vice versa • Given both adjacency matrices • The quality of the match is estimated using the squared Euclidean distance which provides a global measure of the dissimilarity between both graphs

  19. Experimental results • Database • AR face database • This face database was created by Aleix Martinez and Robert Benavente in the Computer Vision Center (CVC) at the U.A.B • It contains over 4,000 color images • 126 people's faces (70 men and 56 women) • 26 image per individual • Images feature frontal view faces • Different facial expressions, illumination conditions, and occlusions (sun glasses and scarf).

  20. Experimental results • Tested the four graph representations • Two parameters have to be set • C is used in the first step of the keypoints matching • C induce a restrictive selection of matching points • K for the three graph representations • It corresponds to the number of neighbors considered in the graph definition • The dissimilarity measure gives a coefficient that can be interpreted as a confidence criterion on the fact that two faces belong to different individuals • Gives the experimental results on the AR face database on a ROC curve

  21. Experimental results • We use the FAR (False Acceptance Rate) and FRR (False Rejection Rate) criteria to analyse the behavior of an algorithm The representation based on a Completely connected graph (CCG) gives the worst result This means that the selection of edges is necessary Mutual k-nearest neighbor graph (MKNG) with k=5 and C=0.8 provides

  22. Conclusion • We proposed in this paper a face authentication method based on a similarity measure • A graph representation based on SIFT invariant keypoints is used • The graph definition allows us to take into account their spatial relationships

More Related