1 / 29

MULTI-OMNIDIRECTIONAL CAMERAS FOR AMBIENT INTELLIGENCE

MULTI-OMNIDIRECTIONAL CAMERAS FOR AMBIENT INTELLIGENCE. Bar ış Evrim Demiröz, Albert Ali Salah, Lale Akarun. 19.04.2012 Sinyal İşleme ve İletişim Uygulamaları Kurultayı. Overview. Dataset collected Baseline tracker Released Videos Annotations Performance Measurer Camera calibrations

eudora
Télécharger la présentation

MULTI-OMNIDIRECTIONAL CAMERAS FOR AMBIENT INTELLIGENCE

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MULTI-OMNIDIRECTIONALCAMERAS FOR AMBIENTINTELLIGENCE Barış Evrim Demiröz, Albert Ali Salah, Lale Akarun 19.04.2012 Sinyal İşleme ve İletişim Uygulamaları Kurultayı

  2. Overview • Dataset collected • Baseline tracker • Released • Videos • Annotations • Performance Measurer • Camera calibrations • Tracker source codes Multi-Omnidirectional Cameras for Ambient Intelligence

  3. Omnidirectional Camera • Omnidirectional camera ~ Large FOV Catadioptric(lens+mirror) PTZ Dioptric (lens) Multiple Images Photos: Jonas Pfeil, Wikimedia commons and `B. Micusik and T. Pajdla, "Estimation of omnidirectional camera model from epipolar geometry" 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings., pp. I-485-I-490, 2003.` Multi-Omnidirectional Cameras for Ambient Intelligence

  4. Catadioptric/Dioptric cameras • Pros/cons • 1 image  • Low resolution near boundaries  • Ideal for small spaces, assisted living, surveillance Title of the presentation

  5. Dataset Multi-Omnidirectional Cameras for Ambient Intelligence

  6. Sample from dataset Multi-Omnidirectional Cameras for Ambient Intelligence

  7. The room Windows Table 1 SideCam Table 2 Chair 1 Chair 2 TopCam Sink Door Multi-Omnidirectional Cameras for Ambient Intelligence

  8. Cameras used • Oncam IPC, IPT • 5 Megapixel • No moving parts Multi-Omnidirectional Cameras for Ambient Intelligence

  9. Scenarios performed • Scenario #1 • Single person • 6 Actions • Sitting • Walking • Drinking • Washing hands • Fainting • Opening/Closing door • 5 performers (10 videos total) Multi-Omnidirectional Cameras for Ambient Intelligence

  10. Scenarios performed • Scenario #2 • 3 people • 5 Actions • Sitting • Walking • Standing • Handshaking • Interested in object • 5 performers (36 videos intotal by exchanging roles) Multi-Omnidirectional Cameras for Ambient Intelligence

  11. Video properties • 640x480 frame size • Bandwidth limitation • ~8 fps • MJPEG format Multi-Omnidirectional Cameras for Ambient Intelligence

  12. Annotations C • Annotation tool usedvatic* • Annotations • Bounding box of subject • State (action) of subject A B * C. Vondrick, D. Ramanan, and D. Patterson, “Efficiently Scaling Up Video Annotation with Crowdsourced Marketplaces,” ECCV 2010, pp. 610–623, 2010. Multi-Omnidirectional Cameras for Ambient Intelligence

  13. Occlusion • Pseudo occlusion: Multi-Omnidirectional Cameras for Ambient Intelligence

  14. Occlusion Multi-Omnidirectional Cameras for Ambient Intelligence

  15. How to evaluate tracker performance? • Multiple Object Tracking Precision/AccuracyMOTP and MOTA* o: object h: hypothesis d: distance btw o: and h c: match count m: misses fp: false positives mme: mismatches t: frame no k: video no i: object no * K. Bernardin and R. Stiefelhagen, “Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics,” EURASIP Journal on Image and Video Processing, vol. 2008, pp. 1–10, 2008. Multi-Omnidirectional Cameras for Ambient Intelligence

  16. Baseline tracker Multi-Omnidirectional Cameras for Ambient Intelligence

  17. Outline System Components Multi-Omnidirectional Cameras for Ambient Intelligence

  18. Background/Foreground Segmentation • Mixture of Gaussians • Shadow removal Multi-Omnidirectional Cameras for Ambient Intelligence

  19. Threshold the areas filter small contours Blob formation • Morphological close • maintain connectivity × × Multi-Omnidirectional Cameras for Ambient Intelligence

  20. FAST*corner detection Feature Point Detection andDescriptor Extraction • BRIEF features • extended for opponent color space • *Features from Accelerated Segment Test Multi-Omnidirectional Cameras for Ambient Intelligence

  21. Blob Matching and Tracking Frame Blob Blob Blob … Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint … … … Keypoint Keypoint Keypoint Multi-Omnidirectional Cameras for Ambient Intelligence

  22. Blob Matching and Tracking Previous n frames Frame NEW FRAME Frame Frame … Blob Blob Blob Blob Blob Blob Blob Blob Blob Blob Blob Blob … … … … Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint … … … … … … … … … … … … Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Multi-Omnidirectional Cameras for Ambient Intelligence

  23. Blob Matching and Tracking • Optimal assignment problem • Hungarian algorithm Multi-Omnidirectional Cameras for Ambient Intelligence

  24. Tracking result example Multi-Omnidirectional Cameras for Ambient Intelligence

  25. Results Multi-Omnidirectional Cameras for Ambient Intelligence

  26. Results Multi-Omnidirectional Cameras for Ambient Intelligence

  27. Results Multi-Omnidirectional Cameras for Ambient Intelligence

  28. Future directions • Utilizing information from multiple cameras • Handling occlusions • Undistorted keypoints • Evaluation protocol for action recognition Multi-Omnidirectional Cameras for Ambient Intelligence

  29. Questions?

More Related