Methods of Deriving Biometric ROC Curves from the k -NN Classifier

1. Methods of Deriving Biometric ROC Curves from the k-NN Classifier Robert S. Zack May 8, 2010

2. Agenda • Introduction to ROC Curves • Classification • Multi-Class Issues and Solutions • New Derivation Methods • Weak and Strong System Training • Use Cases • Search for a Topic • Publications • Dissertation Status • Questions

3. Introduction to ROC Curves • Used for binary decisions • Signal detection – signal / no signal • Medical diagnosis – disease / no disease • Biometric authentication – you are the person you claim to be / you are not • In biometrics the ROC curve varies from FAR=1 & FRR=0 at one end to FAR=0 & FRR=1 at other • FAR = False Accept Rate – the rate an imposter is falsely accepted • FRR = False Reject Rate – the rate the correct person is falsely rejected • ROC Charts are expressed in terms of percentages (0-100%) or probabilities (0-1). These are used interchangeably.

4. Authentication Analogy • Supreme Court – nine judges • Usual procedure – majority required to make decision • Like 9NN needing majority to authenticate a user • ROC Curve – creates many potential procedures • Need 9 votes to make decision (very conservative) • Need 8, 7, 6 votes to make decision (conservative) • Need 5 votes to make decision (majority) • Need 4, 3, 2 votes to make decision (liberal) • Need 1 or even 0 votes to make decision (very liberal)

5. Anatomy of a Biometric ROC Curve • Conservative is too restrictive. • Positive classification requires strong evidence. • Liberal is too open. • Requires weak evidence.

6. Parametric Procedures • Parametric techniques are well studied. • Data follows a normal or Gaussian distribution. • Vary a threshold to obtain the tradeoff between FAR/FRR. • Probability density functions can be calculated without estimation.

7. Parametric ROC Derivation

8. Classification • The k-NN classifier is well studied. • Biometrics classification problems can have many classes. • It is easier to work with a large or unknown population if the data is converted from a multi-class to a two-class decision. • Cha Dichotomy Model.

9. K-NN Nonparametric Classifier • k-NN is nonparametric. • A vector-difference model is used to covert a many class problem into a two class, binary problem. • Uses Euclidean distance k-NN Classification Procedure for k=5, Adapted from Pattern Classification, Duda, et al.

10. Cha Dichotomy Model • Simplifies complexity • Transforms a feature space into a distance vector space. • Uses distance measures. Multi-class to two Class Transformation Process, Adapted from Yoon et al (2005)

11. m-kNN Method • Pure Rank Method. • Evaluate the top 7 NN. • Q is authenticated if # within-class matches is >= decision threshold of 4NN. • Unweighted. All W’s are equal in weight.

12. wm-kNN Method • Rank method weighted by rank order. • Authenticate if W choices are > weighted match (m) • Score varies from 0 to=k(k+1)/2 or5+4+3+2+1 • For every m, FAR/FRR pair or ROC point. • If m=0, FAR=1, FAR=0 …All users accepted. • If m=15, FAR=small, FRR=large, few Q’s accepted.

13. m-kNN and wm-kNN ROC’s LapFree – Weak Training

14. m-kNN and wm-kNN ROC’s DeskFree – Weak Training

15. t-kNN Method • A distance threshold method. • A positive vote is within a distance threshold from the user’s sample. • Uses feature vector space distances only. • At 0, no distance vectors are authenticated. FAR=0, FRR=100%. At t=100, all distance vectors are authenticated. FAR=100, FRR=0.

16. t-kNN Method DeskFree (left) and LapFree (right) Data

17. ht-kNN Method • Weighted vote based on distances to the kNN. • Hybrid of rank method and vector space distances. • For each test sample, the within-class weight (WCW) is calculated based on the distance vectors. DeskFree (left) and LapFree (right) Data

18. New Nonparametric ROC Methods • Need m votes out of k for decision • Pure rank method • Need wm votes for decision, but some judges get more than one vote (weighted method) • Rank method weighted by rank order • A positive vote is within a distance threshold from the user’s sample • Uses feature vector space distances only • Weighted vote based on distances to the kNN • Hybrid of rank method and vector space distances

19. Weak & Strong Training • Weak Training • People used in testing not used in training • Independent sets of users for testing and training • Strong Training • People used in testing also used in training • Usually to augment the different training people • But new difference-vectors used to authenticate • For example, users provide 8 samples – 5 for training and 3 to match against for authentication

20. Weak & Strong Training

21. Use Cases • On-line test taking – Authentication Application • Enroll students at the start of a class. Collect biometric samples. • Authenticate users are who they should be using off-line batch processing. • Corporate Compliance Training/Test Administration • Enroll employees at some point prior to the training or test administration. Collect biometric samples. Refresh them at designated intervals. • Authenticate users are who they should be.

22. Future Work • Real-time authentication. • Accuracy Improvements. • Error Cost Analysis. • Measurement Error.

23. Initial Search for a Topic • Started program in Fall 2008. • Entered DPS with an idea to research a topic in the area of mobile computing. Quickly discarded the idea. • Continued to search for ideas by participating as a Customer for IT691/CS691Projects. Became exposed to Facial and Keystroke Biometrics. • Continued working with Keystroke Biometrics and eventually found a topic with the help of Dr. Tappert.

24. Idea Vetting • The first few presentations of the topic met with a lot of resistance. It took some time to develop the “so what”. • Every Research Seminar was recorded so that I could go back and listen to criticisms. • Participated as co-author to several papers on the subject. Some papers were peer-reviewed and submitted for publication.

25. Publications • [1] J. Abbazio, S. Perez, D. Silva, R. Tesoriero, F. Penna, and R. S. Zack, "Face Biometric Systems," in Student-Faculty Research Day, CSIS, Pace University, White Plains, 2009, pp. C1.1-C1.8. • [2] A. Amatya, J. Aliperti, T. Mariutto, A. Shah, M. Warren, R. S. Zack, and C. C. Tappert, "Keystroke Biometric Authentication System Experimentation," in Student-Faculty Research Day, CSIS, Pace University, White Plains, 2009, pp. C4.1-C4.8. • [3] A. C. Caicedo, K. Chan, D. A. Germosen, S. Indukuri, M. N. Malik, D. Tulasi, M. C. Wagner, R. S. Zack, and C. C. Tappert, "Keystroke Biometric: Data/Feature Experiments," in Student-Faculty Research Day, CSIS, Pace University, White Plains, 2010. • [4] K. Doller, S. Chebiyam, S. Ranjan, E. Little-Tores, and R. S. Zack, "Keystroke Biometric System Test Taker Setup and Data Collection," in Student-Faculty Research Day, CSIS, Pace University, White Plains, 2010. • [5] S. Janapala, S. Roy, J. John, L. Columbu, J. Carrozza, R. S. Zack, and C. C. Tappert, "Refactoring a Keystroke Biometric System," in Student-Faculty Research Day, CSIS, Pace University, White Plains, 2010, pp. B1.1-B1.8. • [6] M. Lam, U. Patel, M. Schepp, T. Taylor, and R. S. Zack, "Keystroke Biometric: Data Capture Resolution Accuracy," in Student-Faculty Research Day, CSIS, Pace University, White Plains, 2010. • [7] C. C. Tappert, S.-H. Cha, M. Villani, and R. S. Zack, "A Keystroke Biometric System for Long-Text Input," International Journal of Information Security and Privacy, Pending Publication, 2010. • [8] R. S. Zack, C. C. Tappert, S.-H. Cha, J. Aliperti, A. Amatya, T. Mariutto, A. Shah, and M. Warren, "Obtaining Biometric ROC Curves from a Non-Parametric Classifier in a Long-Text-Input Keystroke Authentication Study," vol. 268, Pace University, 2009.

26. Questions