1 / 8

Perceptual Interface for Virtual World Control Using Integral Projections

This study presents a method for face detection and tracking in a virtual environment using integral projections. The system allows users to control 3D movement through facial movements. Depth, roll, pitch, and yaw estimations are provided based on integral projections.

nenet
Télécharger la présentation

Perceptual Interface for Virtual World Control Using Integral Projections

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A PERCEPTUAL INTERFACE USING INTEGRAL PROJECTIONS Ginés García Mateos ginesgm@um.es Sergio Fructuoso Muñoz sergiofr@ono.com Dept. de Informática y Sistemas University of Murcia - SPAIN

  2. System overview Face detector and tracker Camera User of the per-ceptual interface MOVEFORWARD/BACKWARDLEFT/RIGHT ROTATE LEFT/RIGHT LOOKUP/DOWN FACE LOCATION:X: 6 Y: 0 Z: 3 FACE ORIENTATION: Roll: 7.7 Pitch: 0 Yaw: 3 FACE MOVEMENT:ΔX: -2 ΔY: 1 ΔZ: 0 3D movement and pose estimation Control signals (virtual world movement) Virtual world rendering

  3. Face integral projections • Definition. Let i(x,y) be an image, and R(i) a region in it: Vertical Integral Projection Horizontal Integral Projection PVR : {ymin, ..., ymax}  R PHR : {xmin, ..., xmax}  R PVR(y) = i(x,y);  (x,y)  R(i) PHR(x) = i(x,y);  (x,y)  R(i) • When applied to human faces, typical patterns of projection appear. • This is used to design a face detector and tracker. Pv(y): Vertical I.P. of the face Ph1(y): Horizon-tal I.P. of eyes’ region Ph2(y): Horizon-tal I.P. of mouth’s region

  4. Face tracking with I.P. Face tracking is a 3-step process. 1. Vertical alignment: compute move-ment and scale in vertical direction 2. Horizontal alignment: compute move-ment and scale in horizontal direct. 3. Orientation alignment: compute orienta-tion of the face Green line:Projection modelRed line:Projection instance • FACE PVFACE(y) Align PVFACE STEP 1 y y Align PHEYES PHEYES(x) EYES STEP 2 x x Align PVEYEi PVEYE1,PVEYE2 EYE1, EYE2 STEP 3 • y • y

  5. Pose estimation (i) • Depth estimation: inversely proportional to the size of the head in the image. Approximated with the eye-to-eye distance. Relative distances 0.674 0.947 1.184 1.636 • Roll estimation: approximated with the perceived angle of the eyes. Roll angles -33.4º -13.0º 13.5º 20.2º

  6. Pose estimation (ii) • Pitch estimation: a heuristic measure is defined, using the vertical I.P. of the face Estimated pitch 19 -2 -9 • Yaw estimation: another heuristic, using the horizontal I.P. of the eyes. Estimated yaw -7 0 11

  7. Virtual environment • The perceptual interface is used to control the movement in a virtual 3D world. • We have used DirectX 9 and OpenCV 3. Virtual 3D world

  8. Results and conclusion • Sample videos available at:http://dis.um.es/~ginesgm/fip/percint.html Conclusions • Depth and roll estimation is very reliable. • Pitch and yaw are less stable.  .. ,10/2004

More Related