Fusion of Face and Iris Biometrics from a Stand-Off Video Sensor

1. Fusion of Face and Iris Biometrics from a Stand-Off Video Sensor Ryan Connaughton Kevin W. Bowyer Patrick Flynn April 16, 2011 Computer Vision Research Lab Department of Computer Science & Engineering University of Notre Dame

2. Biometrics and Multi-Biometrics Biometric Trait Biometric Sample Output Sensor Matcher Multi-Modal Multi-Sensor Multi-Sample Multi-Algorithm Redundancy at any stage is referred to as multi-biometrics 2

3. Fusion in Multi-Biometrics • Fusion: Combining information from multiple sources • Types of fusion: • Signal Level • Feature Level • Score Level • Rank Level • Decision Level 3 3

4. Advantages and Disadvantages • Potential advantages of multi-biometrics: • Increased recognition accuracy • Wider population coverage & lower failure-to-acquire rates • More difficult to spoof • Potential disadvantages: • Increased computation time • Increased acquisition time • Increased sensor cost 4

5. Project Goal • Investigate the feasibility of multi-biometrics based on a single sensor • Specifically, combine multi-sample and multi-modal elements to create a system based on face and iris biometrics • Compare performance of multi-biometric approach to single biometric approach 5

6. Sensors – Iris on the Move (IOM) • Developed by Sarnoff Corp. [1] • Designed for Iris recognition • Stand-off and on-the-move • Array of 3 frontal video cameras • Each frame is 2048 x 2048 px • Average iris diameter is ~120 px • Synchronized NIR illumination Image from K. W. Bowyer, K. Hollingsworth, and P. J. Flynn. Image understanding for iris biometrics: A survey. In Computer Vision and Image Understanding, volume 110, pages 281-307. 2008. 6

7. IOM Frame Example 7

8. Sensors – LG IrisAccess 4000 (LG-4000) • Developed by LG Iris [2] • High-quality iris sensor • Short-range, stationary subjects • Average iris diameter is ~250 px Image from LG Iris Products and Solutions, 2010. URL http://www.lgiris.com/ps/products/irisaccess4000.htm 8

9. LG-4000 Image Example 9

10. Diagram of Approach 10

11. Preprocessing • Stitch and perform histogram matching between corresponding frames • Use template matching to determine translation required to align frames 11

12. Face Detection • Performed on stitched frames • OpenCV version Viola-Jones face detector used [3],[4] • Trained on whole faces • Faces are cropped according to face detector's estimation of size and location 12

13. Eye Detection • Used for iris biometrics and for alignment during face matching • Performed in two phases • Phase 1: Detect eyes in upper quadrants of previously detected faces • Phase 2: Detect eyes in frames where no faces were found • Both phases use template matching approach to search for specular highlights 13

14. Face and Iris Matcher • Face Matcher • Colorado State University's implementation of eigenface [5],[6] • Mahalanobis Cosine: -1 to 1, -1 is perfect match • Iris Matcher • Modified version of Daugman's algorithm [7] • Normalized Hamming Distance: 0 to 1.0, 0 is perfect match 14

15. Fusion Summary • Multi-modal and multi-sample scenario • Test and compare multiple fusion approaches • Score-level • Rank-level • Three approaches: • Min rule • Borda count • Sum rule 15

16. Min Fusion • Multi-sample, uni-modal, score-level fusion MinIris = Min{ Ii,j | i=1...n, j=1...G } MinFace = Min { Fi,j | i=1...m, j=1...G } Ii,j = HD between i-th probe iris and j-th gallery iris Fi,j = Mahalanobis distance between i-th probe face and j-th gallery face n,m = number of irises and faces detected G = number of gallery subjects 16

17. Borda Fusion • Multi-sample, multi-modal or uni-modal, rank-level fusion • For each probe biometric sample • Sort gallery subjects by match score (best to worst) • Cast votes for the top v-ranked gallery subjects • BordaLinear: VoteWeightn = v + 2 – n • BordaExp: VoteWeightn= 2v-n • Gallery subject with the most votes is the best match for that probe video • Three variations: BordaIris, BordaFace, and BordaBoth 17

18. Sum Fusion • Multi-sample, multi-modal or uni-modal, score-level fusion Ii,k = HD between i-th probe iris and k-th gallery iris FNormi,k = Normalized Mahalanobis distance between i-th probe face and k-th gallery face n,m = number of irises and faces detected α,β= weights assigned to face and iris modalities 18

19. Dataset • Collected 1,886 IOM video sets, spanning 363 subjects • Ranged from 1 to 15 probe videos per subject • Iris gallery consisted of one left eye and one right eye for each subject • Acquired with an LG-4000 • Face gallery consisted of one full face image for each subject • Manually selected and annotated from stitched IOM frames • Earliest IOM video with full face available was used to generate gallery image • Videos used to generate gallery images were not included in probe set 19

20. Detection Results 20

21. Face Matching Results Mean match score: -0.281 (σ = 0.213) Mean non-match score: 0.000 (σ = 0.676) Independent rank-one: 51.6% (5073/9833) 21

22. Iris Matching Results Mean match score: 0.398 (σ = 0.053) Mean non-match score: 0.449 (σ = 0.013) Independent rank-one: 46.6% (13556/29112) 22

23. Rank-One Recognition Rates 23

24. Comparison Summary 24

25. Conclusions • Investigated fusion of face and iris biometrics from a single sensor • Conducted multi-modal experiments on a genuine dataset of 1886 videos of 363 subjects • Combined multi-modal and multi-sample biometrics, as well as score-level and rank-level fusion • Implemented the proposed multi-biometric workflow on a stand-off and on-the-move sensor • Thus far, the best tested multi-modal approach yielded an increase of 5.4% in rank-one recognition over uni-modal approach 25

26. Acknowledgments & Questions [1] J. Matey, O. Naroditsky, K. Hanna, R. Kolczynski, D. LoIacono, S. Mangru, M. Tinker, T. Zappia, and W. Zhao. Iris on the Move: Acquisition of Images for Iris Recognition in Less Constrained Environments. In Proceedings of the IEEE, volume 94, pages 1936-1947. November 2006. [2] LG Iris. LG Iris Products and Solutions, 2010. URL http://www.lgiris.com/ps/products/irisaccess4000.htm. [3] G. Bradski and A. Kaehler. Learning OpenCV. O'Reilly Media, Inc., 2008. [4] P. Viola and M. Jones. Rapid Object Detection Using a Boosted Cascade of Simple Features. In 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2001), volume 1, pages 511-518, 2001. [5] Colorado State University. Evaluation of Face and Recognition Algorithms, 2010. URL http://www.cs.colostate.edu/evalfacerec/algorithms6.html. [6] M. Turk and A. Pentland. Face Recognition Using Eigenfaces. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 1991), volume 1, pages 586-591, June 1991. [7] J. Daugman. How Iris Recognition Works. In 2002 International Conference on Image Processing, volume 1, pages 33-36, 2002. Datasets used in this work were acquired under funding from the National Science Foundation under grant CNS01-30839, by the Central Intelligence Agency, and by the Technical Support Working Group under US Army Contract W91CRB-08-C-0093. Current funding is provided by a grant from the Intelligence Advanced Research Projects Activity. 26