1 / 15

VUT Brno meeting data collection for M4

VUT Brno meeting data collection for M4. Petr Jenderka , Stanislav Sumec, Igor Pot ůček, Petr Mot líček, Jan Černocký, Pavel Zemčík. Brno University of Technology, Czech Republic. Goals. Provide M4 and follow-ups with more data, complementary to ICSI and IDIAP

flower
Télécharger la présentation

VUT Brno meeting data collection for M4

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. VUT Brno meeting data collection for M4 Petr Jenderka, Stanislav Sumec, Igor Potůček, Petr Motlíček, Jan Černocký, Pavel Zemčík Brno University of Technology, Czech Republic M4 meeting Delft

  2. Goals • Provide M4 and follow-ups with more data, complementary to ICSI and IDIAP • Different speakers, different accent (heavily accented CZEnglish) • Setup of a secondary M4 meeting room: low-cost, mobile, built from off-the-shelve products, easy to install and operate. • Plans for meeting recordings • Plans for annotations. M4 meeting Delft

  3. The rooms • IDIAP room very valuable but expensive, difficult to assemble/setup and impossible to bring outside IDIAP Need for cheaper, lighter, and “buyable” room • Our meeting room: • recordings of up to 4 persons • 4 lapel mikes, no mike arrays • one camera with hyperbolic mirror allowing for almost 360 degree view. • audio/video recording on one notebook. M4 meeting Delft

  4. Setup M4 meeting Delft

  5. 4 people around a mirror M4 meeting Delft

  6. M4 meeting Delft

  7. Original wrapped image M4 meeting Delft

  8. Hardware • 4 lapel microphones Sennheiser MKE 2 P-C (same as in IDIAP) • 2 mixers Behringer EURORACK UB1204-PRO providing gain control and phantom power supply (actually 90% of mixers are not used, but they’re cheaper than special mike pre-amps with phantom feeders…) • two Hi-Fi PC-MCIA sound-cards VX-pocket (we know them from the SpeeCon project) M4 meeting Delft

  9. Hardware (2) • digital camera SONY CLIP 345 • hyperbolic mirror by NEOVISION (Prague) • FireWire interface • The camera provides 2 additional (global) audio channels. • The cost: 4.2 kEUR w/o the notebook M4 meeting Delft

  10. Recording • a home-made (S. Sumec) tool, based on Direct-show libraries, under Windows. • post-processing tool (I. Potúček) for unwrapping video to panoramic format of view. • All audio and video streams are stored in real time: two audio wav files with four audio channels, one DV-compressed video avi file with video and stereo audio stream. • Plan for M4: >30 hours of meetings M4 meeting Delft

  11. Unwrapped image M4 meeting Delft

  12. Annotations – major meeting events • annotations using the „annotation editor“ by Staňa Sumec (audio, video, hotkeys, XML). • This annotation will be done internally by pre- and post-grads at UPGM. • Plan: >20 hours M4 meeting Delft

  13. Annotations - speech • ETCA Transcriber with ICSI MultiChannel extension. • An annotator hired for speech annotations, just beginning to be operational. • Plan: >10 hours of speech annotations. • Both annotations: quality check protocols? (quite an issue in our DB projects SpeechDat-E and SpeeCon!) M4 meeting Delft

  14. Current work and future • Recording, annotations, integration to the Media Server mmm.idiap.ch • Still quite some issues to solve (position of the mirror, lighting in the room – what can be done on the resulting video?) • Extensions – PowerPoint and Acrobat Reader slide/change timing grabber, T0 must be synced with the recording software • … and research with the data (phoneme recognition, mouth area detection, speaker activity detection) – where does the simplified setup degrade performances? M4 meeting Delft

  15. Questions, comments ?THE DEMO Please have a look during the break :-) M4 meeting Delft

More Related