1 / 14

Facial Tracking and Animation

Facial Tracking and Animation. Todd Belote Bryan Harris David Brown Brad Busse. Problem Background. Speech driven facial animation Correlate captured facial movements to audio patterns Capture facial movements Analyze corresponding audio. Goals and Objectives.

juniper
Télécharger la présentation

Facial Tracking and Animation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Facial Tracking and Animation Todd Belote Bryan Harris David Brown Brad Busse Facial Tracking and Animation

  2. Problem Background • Speech driven facial animation • Correlate captured facial movements to audio patterns • Capture facial movements • Analyze corresponding audio Facial Tracking and Animation

  3. Goals and Objectives • Develop an inexpensive, robust real-time system to track facial motion and process corresponding audio. The system must: • Cost around $1000 • Run on a personal computer • Allow for long periods of data acquisition • Handle head movements • Recover from point occlusion • Output only necessary information Facial Tracking and Animation

  4. System Functionality • Capture Live Data • Record Video • Capture From Video • Generate FAP from output file • Process audio of .wav file Facial Tracking and Animation

  5. POINT INITIALIZATION POINT TRACKING FAP GENERATION DATA ACQUISITION AUDIO PROCESSING System Description • Top level system organization • Illustrates data flow • Functional block division Facial Tracking and Animation

  6. CAPTURE CARD EH WAV FILE AUDIO PROCESSING MICROPHONE CAMERA EH VIDEO PROCESSING FRAMEGRABBER AVI MOVIE FILE SW TIMER Data Acquisition Video Data – 320x240 at 30 fps (bmp) Audio Data – 16 bit at 16kHz (wav) Facial Tracking and Animation

  7. Point Initialization and Tracking • Process frame data • Locate all areas in the frame which constitute a point • Identify points • Initialization – algorithm to determine identity • Tracking – algorithm to match current frame points with previous location of identified points Facial Tracking and Animation

  8. Point Transform Approach: Criminisi et al. • Maps any arbitrary quadrilateral onto any other • This can account for all six degrees of freedom as well as perspective distortion, greatly simplifying the computation required to reorient the face • When using an orientation square that encompasses most of the face, this algorithm can be made as accurate as necessary Facial Tracking and Animation

  9. Point Transform: Demo Actual Camera Data Re-oriented Processed Data Facial Tracking and Animation

  10. FAP Generation and Audio Processing • FAP Generation • Process text file of point movement • Output FAE Engine compliant file of point displacements • Audio Processing • LPC • MFCC • Pitch • Power Facial Tracking and Animation

  11. Environmental and Health Considerations • All hardware is off the shelf • No harm from infrared light • No harm from other products • Eg. Reflective markers Facial Tracking and Animation

  12. Social, Political and Ethical Considerations • Provide low cost audiovisual capture • Increase research in field by removing cost barrier • Further advances • Eg. Phone for the deaf • No Ethical Issues • No Political affects Facial Tracking and Animation

  13. Economics and Sustainability • No economies of scale due to narrow scope • IBM PupilCAM is hard to locate and therefore sustainability with current hardware is issue • Other cameras could provide the same function Facial Tracking and Animation

  14. Thanks For the Contributions Facial Tracking and Animation

More Related