New work - package WP5: Multimodal Processing and Interaction MUSCLE JPA3

1. New work-package WP5:Multimodal Processing and InteractionMUSCLE JPA3 Leaders: Petros Maragos, ICCS-NTUA Alexandros Potamianos, TSI-TUC

2. Outline of new WP5 Work in JPA3 • T1. Book on Multimodal Processing and Interaction • T2. Audio-Visual Speech Analysis and Recognition • T2.1Audio-Visual Feature Extraction and Fusion • T2.2: Dynamic Models for AV-ASR, Evaluation • T2.3: Audio-Visual to Articulatory Speech Inversion • T3. Multimodal Integration for MM Analysis & Recognition • T3.1: Video Analysis & Integration of Asynchronous Time-evolving Modalities • T3.2: Multimodal Saliency • T3.3: Integrated Multimedia Content Analysis • T4. Interfaces to Multimedia • T4.1: Multimodal Dialogue Interfaces • T4.2: Eye-tracking Interfacesfor Information Retrieval • T4.3: Mobile Interfaces • T5. Coordination of research and Dissemination of results MUSCLE Plenary, Dec. 2006, France

3. Next steps & Proposed Research Orientations • Merge research tasks of JPA1-JPA2/WP6+WP10  JPA3/WP5 Notify partners. • The main axes of research in Multimodal Processing, Interaction, Interfaces are fundamental and still timely (multimodal information fusion for multimedia understanding). • Focus on specific areas • Need for even closer collaboration between the WP partners on specific tasks of common interest (jointly planned research agendas, joint papers, fellowships, joint proposals for new added-value in-depth projects). • Contributions of each partner to deliverables should be more clearly specified to help people focus their research towards specific WP goals and collaborate with other groups. • Further develop Synergies with other WPs. • E-teams • Book MUSCLE Plenary, Dec. 2006, France

4. e-Teams: Goals & Objectives • E-Team 10: Audio-Visual Speech Analysis & Recognition • AV Feature Extraction and Feature Fusion • Dynamical Models for AV-ASR, Evaluation • Audio-Visual to Articulatory Speech Inversion. • E-Team 11: Multimodal Processing & Multimedia Understanding • Video Analysis and Integration of Asynchronous Time-evolving Modalities • Audio-Visual Attention Modeling and Salient Event Detection • Integrated Multimedia Content Analysis • E-Team 12: Multimodal Interfaces • Multimodal Recognition and Dialogue Systems • Mobile Services • Novel Interfaces (Eye-tracking) MUSCLE Plenary, Dec. 2006, France

5. BOOK • Title: Multimodal Processing and Interaction: Audio, Video, Text • Contents: • State-of-Art Reviews of WP6 + WP10 (updated) • Contributed Research Chapters: New Work • Agenda: • Scope and Thematic Areas discussed during Audio-Conf & Meetings • Each interested participant emails preliminary title + abstract • Table-of-Contents of selected chapters is discussed with all participants • Publisher is contacted MUSCLE Plenary, Dec. 2006, France

6. Dissemination and Publicity • Necessary to attract better research and collaboration. • Website • Showcase Demonstrations of prototype systems sampling MUSCLE research. • Organize MUSCLE-related Special Sessions at related Conferences: e.g., ICASSP, EUSIPCO, ICME, ICSL, ICIP, ECCV, ICCV • Publish Updated State-of-the-Art report. • Present & Publishjoint papers at conferences and journals. • Book. MUSCLE Plenary, Dec. 2006, France

7. Multimodal Research and Systems are Central in MUSCLE Research Challenges & Goals MUSCLE Plenary, Dec. 2006, France

8. MMSP 2007 • IEEE Workshop on Multimedia Signal Processing in 2007 • co-organized by A. Potamianos (GC) and P. Maragos (TC) • Where: Chania, Crete • When: Oct 1-3, 2007 • Synergy with MUSCLE • Multimedia a central research area in MUSCLE • MUSCLE sponsors four student best paper MMSP 2007 awards • Possibility of organizing a MUSCLE special session (spec. session proposal deadline March 9th) • Discussion of hosting a Plenary MUSCLE meeting in Chania as a satellite workshop to MMSP MUSCLE Plenary, Dec. 2006, France

9. Multimedia Interaction Modeling of multimodal perception Modeling of multimedia interaction Human-human and human-computer dialog Multimodal fusion and fission Multimedia Communication Multimedia peer-to-peer communication Multimedia wireless communication QoS in multimedia communication Cross-layer design for multimedia communication Supporting Methods and Systems Single- and multimedia analysis Single- and multimedia compression Single- and multimedia security Single- and multimedia indexing Single- and multimedia search Emerging Applications Distributed multimedia Entertainment and gaming eHealth and telemedicine Advanced Implementations Implementations on graphics cards Implementations on multimedia processors Wearable multimedia implementations CFP: Multimedia Interaction and Communication MUSCLE Plenary, Dec. 2006, France

10. The Venue • Panorama Hotel • Location: 10 km from Chania on the beach • Conference rooms with capacity up to 800 • Easy access to restaurants nearby and town of Platanias • Numerous alternative hotels nearby MUSCLE Plenary, Dec. 2006, France

11. MUSCLE Plenary, Dec. 2006, France

12. MUSCLE Plenary, Dec. 2006, France

13. MMSP 2007 Important Dates • Special Sessions (contact the respective chair by) • March 9, 2007 • Papers (full paper, 4 pages, to be received by) • April 6, 2007 • Notification of acceptance by • June 8, 2007 • Camera-ready paper submission by • July 6, 2007 http://www.mmsp2007.org/ Posters and CFP available! MUSCLE Plenary, Dec. 2006, France