1 / 17

~ Multimodal Communication ~

~ Multimodal Communication ~. HOW TO: From raw data to data annotation. Raw data: video file. from video camera to the computer TASX compatible format: AVI for help with this: AVZ (Audio-visuelles Zentrum) to be found on N6, N7 http://www.uni-bielefeld.de/avz/.

rafiki
Télécharger la présentation

~ Multimodal Communication ~

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ~ Multimodal Communication ~ HOW TO: From raw data to data annotation

  2. Raw data: video file • from video camera to the computer • TASX compatible format: AVI for help with this: AVZ (Audio-visuelles Zentrum) to be found on N6, N7 http://www.uni-bielefeld.de/avz/

  3. Editing the video file with VirtualDub cutting the video/ making selections

  4. Editing the video file with VirtualDub saving changes: File: Save as AVI select file name & location

  5. Extracting audio from video with VirtualDub saving audio stream: File: Save WAV select file name & location

  6. Data output • digitized video file, e.g. 9-11.avi • digitized sound file, e.g. 9-11.wav Why extract the sound file from the video file? --> separate speech description in Praat (or other tools, for that matter)

  7. Speech annotation with Praat: Individual steps • load .wav file into Praat, • set up TextGrid (annotation tiers), and • EDIT; • annotate speech file according to individual needs (granularity of segmentation, means of transcription, ...) • write TextGrid to text file (STRG-S), e.g. 9-11.TextGrid

  8. Speech annotation with Praat:Illustration

  9. From Praat file to TASX file applying script (praat-label2tasx) to Praat file for conversion from .TextGrid to .xml in order to make it compatible with TASX (output: e.g. 9-11.xml)

  10. Confused? Brief review on previous steps • editing of video file (file.avi) • extraction of sound stream from video file (file.wav) • annotation of speech in Praat (or some other tool...) (file.TextGrid) • conversion of file.TextGrid into TASX compatible format (file.xml)

  11. What is TASX? • TASX - Time Aligned Signal data eXchange (XML based) • tool for the annotation of multimodal (audio & video) data (cf. Praat for audio only) • Source: http://tasxforce.lili.uni-bielefeld.de

  12. The TASX annotator • a TASX-annotated corpus consists of a set of sessions • each session holds an arbitrary number of descriptive tiers or layers • each layer consists of a set of separated events • each event holds some kind of textual information (label) and is linked to the primary audio or video data by means of two time stamps (marking the beginning and the end of an interval)

  13. Terminology corpus, plural corpora: A collection of linguistic data, either compiled as written texts or as a transcription of recorded speech. The main purpose of a corpus is to verify a hypothesis about language - for example, to determine how the usage of a particular sound, word, or syntactic construction varies. [...] (cf. Crystal, David. 1992. An Encyclopedic Dictionary of Language and Languages. Oxford, 85)

  14. Interface of the TASX annotatorIllustration tiers or layers segments or events labels: e.g. “mom“

  15. Getting started... Part I • Start the TASX annotator. • Load primary video: File - Load primary video or STRG-W (.avi file) • Load audio file (.wav file, optional; loading audio file causes TASX annotator to generate oscillogram) • Import speech annotation: File - Import from - TASX (.xml file) • In case there is already an existing annotation file (.tbf file), file can simply be loaded into the tool: File - Open (new format) or STRG-O

  16. Getting started... Part II • File - Merge (new format) merges two separate annotation files • Add (an) extra tier(s): Tier - New tier or Shift-N • Rename extra tier(s): • Activate tier to be renamed by mouseclick (changes to green) • Tier - Rename tier or Shift-R • Type in new tier name • Save complete file: File - Save as... (new format) or STRG-S resulting output file: file.tbf

  17. Getting started... Part III • For further details, see the manual: http://www.spectrum.uni-bielefeld.de/~thies/TASX_ann.doc ... And now, let‘s get started!

More Related