Eye/gaze tracking in video; identify the user’s “focus of attention”

1. Eye/gaze tracking in video; identify the user’s “focus of attention” Mihaela Romanca – Technical University of Cluj-Napoca Peter Robert - Technical University of Cluj-Napoca Vilius Matiukas - Vilnius Gediminas Technical University Brigitta Nagy – University of Debrecen

2. Introducing the team SSIP 2009

3. Mihaela Romanca Student from Technical University of Cluj-Napoca Hobbies: Sports and ecology E-mail: mihar_bv@yahoo.com SSIP 2009

4. RobertPeter Masters student from Technical University of Cluj-Napoca Hobbies: PC games, football and movies/music E-mail: p_robi86@yahoo.com SSIP 2009

5. Vilius Matiukas PhD student from Vilnius Gediminas Technical University, Faculty of Electronics, Department of Electronic Systems Hobbies: Image Processing and fishing E-mail: vilius.matiukas@el.vgtu.lt SSIP 2009

6. Brigitta Nagy Student from University of Debrecen, Faculty of Informatics Hobbies: Image processing, Wing-Tsun Kung-fu, Reading and Puzzles E-mail: stefanie867@yahoo.co.uk SSIP 2009

7. Test subject Uneducated peace of paper Hobbies: Staring at the same direction. Address: computer laboratory SSIP 2009

8. Problem Description • Input: video of a user sitting in front of the computer • Goal: Detect the focus of attention and the modification of the region of interest of the user. SSIP 2009

9. Equipment and software • Genius Slim 321c webcamera. • Language: C# • IDE: Microsoft Visual Studio 2005 • EMGU CV: Wrapper for C# of OpenCV SSIP 2009

10. Tasks to do • Face Detection • Detection of the eyeregion • Pupil Detection • Eye Corner Detection • Determine the focus of attention SSIP 2009

11. 1. Face Detection • We used Haar-like features for face detection. • Haar-like features are digital image features used in object recognition. • Then we reduced the face region and split it to region of eyes. SSIP 2009

12. Example • Our test subject SSIP 2009

13. 2. Detection the region of the eye • To detect the region of eyes we used also Haar-like features. • Then contrast enchancement on the detected eye region was applied. SSIP 2009

14. Example • The test subject Approximation With Haar-like features SSIP 2009

15. 3. Pupil detection • Circular Hough transformation was applied for detection of the pupil. • The Hough transform is a feature extraction technique. • The classical Hough transform was concerned with the identification of lines in the image, but later the Hough transform has been extended to identifying positions of arbitrary shapes, most commonly circles or ellipses. SSIP 2009

16. 4. Eye Corner Detection • We choose the one closest corner to the nose. SSIP 2009

17. Calibration for gaze detection • Wait until the user sits in a position, where 80% of the frames detect the iris center and the corner also. • Put circles in the center and the four extremities of the screen, and wait until at least 15 pupil and eye corners are detected in both region of eye. • Calculate the average of eye corner and center coordinates in all the positions (center, topleft, topright…). SSIP 2009

18. 5. Focus of attention • As the users moves the eyes the mouse cursor moves in the corresponding direction. SSIP 2009

19. Statistics The numbers represent the variance of coordinates during calibration. SSIP 2009

20. Future development • Imitating left and right mouse clicks with blink detection • Recognition even when face is in different angle • Expression detection for different focus regions • Higher precision for full control for people with disabilities SSIP 2009

21. Thank you! SSIP 2009