220 likes | 323 Vues
This paper introduces a face authentication method using similarity measures and SIFT descriptors for keypoint matching. Scale Invariant Feature Transform (SIFT) is applied for feature extraction and matching, with dissimilarity measured by weighted bipartite graph algorithms like the greedy, Hungarian, and hybrid methods. The process involves detecting key points, localizing them, assigning orientations, and creating descriptors for matching using dissimilarity measures between keypoint sets to optimize global matching scores. A combination of greedy and Hungarian methods is proposed for robust and accurate face authentication.
E N D
Similarity-based matching for face authentication Christophe Rosenberger Luc Brun ICPR 2008
outline • Introduction • Image description • Dissimilarity measure • Pattern modeling • Experimental results
Introduction • Propose in this paper a face authentication method based on a similarity measure • Find interest key points • SIFT descriptor is used • A graph is then built where nodes correspond to these keypoints • Model the authentication problem as a graph matching process
Image description • SIFT (Scale Invariant Feature Transform) • Applied locally on key-points • Based upon the image gradients in a local neighborhood • SIFT detector • Scale-space extrema detection • Keypoint localization • SIFT descriptor • Orientation assignment • Keypoint descriptor
SIFT detector • Detection of scale-space extrema • DoG(Difference-of-Gaussian) filter for scale space • Keypoint localization (local extrema) X is selected if it is larger or smaller than all 26 neighbors
SIFT detector & descriptor • Accurate keypoint localization • Reject points with low contrast (flat) • Elimination edge responses • Orientation assignment • To make the feature invariant to rotation • Find local major orientation • Orientation histogram for a window size • Local image descriptor • 16x16 array of locations in scale space • Create array of orientation histograms • 8 orientations x 4x4 histogram array (128D) Gradient magnitude Gradient orientation
Dissimilarity measure • Each individual is described by a set of invariant features Y (I) = {ki = (si, xi, yi)}, i = 1 : N(I) • Si is the SIFT invariant descriptor • (xi, yi) its position in the original image I • N(I) the number of detected keypoints in the image I • Database image • {Y (Ii), i = 1 : M} • Dissimilarity between two sets of points such as Y (I1) and Y (I2) • Often modeled as a weighted bipartite graph matching problem • Greedy algorithm • Hungarian method • Hybrid method
Greedy algorithm • Weighted bipartite graph G(U,V,E,W) • Given the graph G , scan the set of edges and find an edge (u,v) with the smallest weight • Match u with v and remove the edges incident on u and those incident on v • Repeat the above step until every node in U is matched to some node in V 3 3 10 12 5 5 20 25 6 4 9 4
Greedy algorithm • The greedy algorithm does not always produce a matching of minimum weight • Hungarian method • A well-known algorithm for finding a minimum weight matching in a bipartite graph • Node in U by u1, u2,… un • In V by v1, v2,… vn • x ij= 1 if ui is matched with vjand 0 otherwise subject to constraints 1 A A C C 16 16 0 0 B 2 D B D Each node in U is matched to exactly one node in V and vice versa
Hungarian method - Example Step 1 First for each row we subtract the row minimum from the rest of the row R1 B1 R2 B2 R3 B3 R4 B4 Step 2 Then for each column we subtract the column minimum from the rest of the column
Hungarian method - Example R1 B1 Step 4 1.Find the min value not covered (B3-R1) 2.subtracted from each uncovered entry and added to each entry covered by both a vertical and horizontal line (B1-R2 & B1-R4) 3. Step 3 is then repeated Step 3 Cover the min number of rows or columns required to cover all of the zeros in the matrix R2 B2 R3 B3 R4 B4 If the # of line = the # of rows/columns in the matrix then we are finished Otherwise, we have to go to the next step 3 Result: Blue1 - Red3Blue2 - Red2Blue3 - Red1Blue4 – Red4 4 13 2
Hybrid method • We formulated the point matching problem in terms of minimum weight matching in a bipartite graph • In a real world situation, this is not always appropriate • Feature points extracted from two corresponding images • First image are denoted by a and b • Second image are denoted by c and d One might argue that since b and c both have the same feature value of 100,we should match them together and completely ignore a and d
Hybrid method • Feature points forming tight clusters in the feature space can be avoided by a simple combination of the greedy method and the Hungarian method • Combined algorithm • We first apply the Hungarian method to get a matching M1 • Apply the greedy algorithm to get a matching M2 • form their intersection M=M1∩M2 Greedy algorithm Combined matching A bipartite graph Hungarian method
Dissimilarity measure • Such a weighted bipartite graph matching method • Optimizes a global matching score • May thus be sensible to outliers • We thus rather use the following matching method • Given two points and , we say that x is associated to y iff : • C is an arbitrary threshold • d(., .) denotes the Euclidean distance between SIFT descriptors • y’ denotes any point of Y (I2) whose distance to x is minimal but greater than d(x,y)
Dissimilarity measure • The significance of the necessary gap between d(x, y) and d(x, y’) is encoded by the constant C • y is associated to x • x is the closest point from y among Y (I1) • second smallest value of this distance d(y, x’) satisfies d(y, x) < Cd(y, x’) • X is matched to Y • iff x is associated to y and y is associated to x • If Y (I1) and Y (I2) do not define any matching point • Both sets have an infinite distance • Otherwise • The set of matching points M1 of Y1 and M2 of Y2 may be computed sequentially • If we cannot find two couples of points within Y (I1) × Y (I2) with a similar distance, the relationship “is matched to “ induces a bijective mapping Φ from M1 to M2
Dissimilarity measure • First step • The above procedure classify both sets Y (I1) and Y (I2) • Dissimilar (case M1 = M2 =ø) • A set of N = |M1| = |M2| couples (x, Φ(x)) of matched points • Second step • Take into account the spatial relationship of the points among M1 & M2 • Use one of the four pattern representations • Using the locations of each matched keypoint • Weighted graph • Edges are weighted by the similarity of the adjacent points • The matching between M1 and M2being encoded by the application Φ • Encode the adjacency matrices of both graphs • A line or a column encoding the adjacency relationships • Point x of the first matrix • Φ(x) in the second matrix Graph Adjacency matrix
Pattern modeling • Spatial relationship between keypoints • Using graph representations • Connect keypoints by considering their ”local neighborhoods” • Each keypoint corresponds to a vertex in the neighborhood graph which connects “close” vertices • Epsilon-neighborhood graph (ENG) • Two vertices are connected if the distance of the corresponding keypoints is less than epsilon • A completely connected graph (CCG) • epsilon greater than the maximal distance between keypoints.
Pattern modeling • Symmetric k-nearest neighbor graph (SKNG) • Two vertices V1, V2are connected if V1 is among the k-nearest neighbors of V2 or vice versa • Mutual k-nearest neighbor graph (MKNG) • Two vertices V1, V2 are connected if V1 is among the k-nearest neighbors of V2 and vice versa • Given both adjacency matrices • The quality of the match is estimated using the squared Euclidean distance which provides a global measure of the dissimilarity between both graphs
Experimental results • Database • AR face database • This face database was created by Aleix Martinez and Robert Benavente in the Computer Vision Center (CVC) at the U.A.B • It contains over 4,000 color images • 126 people's faces (70 men and 56 women) • 26 image per individual • Images feature frontal view faces • Different facial expressions, illumination conditions, and occlusions (sun glasses and scarf).
Experimental results • Tested the four graph representations • Two parameters have to be set • C is used in the first step of the keypoints matching • C induce a restrictive selection of matching points • K for the three graph representations • It corresponds to the number of neighbors considered in the graph definition • The dissimilarity measure gives a coefficient that can be interpreted as a confidence criterion on the fact that two faces belong to different individuals • Gives the experimental results on the AR face database on a ROC curve
Experimental results • We use the FAR (False Acceptance Rate) and FRR (False Rejection Rate) criteria to analyse the behavior of an algorithm The representation based on a Completely connected graph (CCG) gives the worst result This means that the selection of edges is necessary Mutual k-nearest neighbor graph (MKNG) with k=5 and C=0.8 provides
Conclusion • We proposed in this paper a face authentication method based on a similarity measure • A graph representation based on SIFT invariant keypoints is used • The graph definition allows us to take into account their spatial relationships