1 / 18

Animal Tracking for Behavior Modeling

Animal Tracking for Behavior Modeling. James M. Rehg Georgia Tech (standing in for Tucker Balch). People. PI Tucker Balch Faculty Jim Rehg, Computer Science Collaborator on animal tracking Aaron Bobick, Computer Science Collaborator on animal behavior modeling Bruce Walker, Psychology

jirair
Télécharger la présentation

Animal Tracking for Behavior Modeling

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Animal Tracking for Behavior Modeling James M. Rehg Georgia Tech (standing in for Tucker Balch)

  2. People • PI • Tucker Balch • Faculty • Jim Rehg, Computer Science • Collaborator on animal tracking • Aaron Bobick, Computer Science • Collaborator on animal behavior modeling • Bruce Walker, Psychology • Collaborator on aquarium project • Atsushi Nakazawa, Computer Science (Osaka Univ., Japan) • Collaborator on animal tracking • PhD Students • Michael Novitzky • Jin Lee • Matthew Flagg

  3. Goals • Track social animals reliably under natural conditions (in vivo) from video • Estimate behavioral parameters from tracking data • Construct executable models of social animal behavior • Develop a biologically-inspired protocol for dynamic team formation

  4. Year 1 Overview • Goal • Develop reliable multi-target tracking algorithms for animals in video • Approach • Jointly estimate segmentation and motion of a nonrigid, deformable target • Key requirements • Reliably estimate target shape over time, to support behavioral analysis • Minimize the amount of human effort required

  5. Challenges • Accurate segmentation of target • Set-point tracking is not sufficient • Track reliably with significant camera motion • Background subtraction is not sufficient • Handle a wide range of animals • Scalable solution for the animal kingdom  • Track multiple interacting targets • Multiple instances of target type (e.g. wolf pack) • Occlusions (with self, other targets, and background) • Track long sequences, track across cuts

  6. Approach • Basic research in video object segmentation and tracking • Modular software architecture • Easily change features, models, and solver • Testing in two scenarios • Aquarium Monitoring • Long sequences, static camera, some modeling effort • Tracking Animal Planet videos • Shorter sequences, moving camera, minimal human effort • Preliminary application • Accessible Aquarium Project

  7. Tracking App • Basic Trackers: • Contour feature w/ Iterated Closest Point • Color feature w/ Mean Shift

  8. Color Histogram Tracker • Fish Model: • Appearance model • For each species of fish, multiple HSV Color Histograms on Image patches • Off-line Model Selection by Human • Detection and Tracking: • Histogram-based mean shift approach • Maximization of Bhattacharyya Coefficient between Observation and Model • Selection of a model with the highest coefficient and update track

  9. Histogram-based Model Selection 0.6 0.5 0.7 * 0.8 0.3 Input Image Model Image Patches (to build histogram) Best Model Image Patch after Mean-shifting Similarity between shifted & model patches

  10. Accessible Aquarium Project • Provide a meaningful and informative aquarium experience for visually-impaired or blind visitors • Approach • Track the movement of individual fish within tank • Sonify the fish movement • Example: 65 gallon marine aquarium • Track yellow tangs and blue chromis • Music structure is Bach chorale • Each fish type is same instrument, different registers • Movement speed mapped to note density (tempo) • Horizontal dimension is stereo, vertical is timbre

  11. Example • Play video

  12. Tracking in the Video Volume

  13. Graphcut Tracking

  14. Tracking Results

  15. Social Game Retrieval

  16. Overview

  17. Summary of Progress • Robust long-life tracking of multiple targets (fish) • Tracking under controlled (but realistic) conditions • Virtual Aquarium Project • Novel experience of animal behavior via sonification • New state-of-art motion segmentation algorithm • Accurate segmentation of (shorter) video sequences under a wide range conditions

  18. Year 2 Plans • Make general tracking method work for longer sequences • Develop integrated tracking application tool for biologists • Connect to biologists more effectively (what to say about this?) • Start developing behavior models, how?

More Related