SlideServe
Télécharger Obtenir Pro
Menu
Accueil Pro
Préférences
ApparenceLight Aide et commentaires
© SlideServe · v2026

Multimodal Emotion Recognition

saul avatar
saul
19 diapositives · 1.1K Views · Jul 26, 2014

Multimodal Emotion Recognition. Colin Grubb Advisor: Nick Webb. Motivation. Multimodal fusion Research looking at audio, visual, and gesture information Feature Level vs. Decision Level. Previous Research.

Multimodal Emotion Recognition - slide 1
01 / 19Multimodal Emotion Recognition
1 / 17

Télécharger la présentation

Multimodal Emotion Recognition
Une image / un lien ci-dessous est fourni (tel quel) pour télécharger la présentation

Politique de téléchargement : le contenu du site vous est fourni TEL QUEL pour votre information et votre usage personnel ; il ne peut être vendu, concédé sous licence ni partagé sur d’autres sites sans l’accord de son auteur.Le contenu vous est fourni TEL QUEL, pour votre information et votre usage personnel uniquement. Téléchargez la présentation en cliquant sur ce lien. Lors du téléchargement, si vous ne parvenez pas à télécharger une présentation, l’éditeur a peut-être supprimé le fichier de son serveur.Pendant le téléchargement, si vous ne parvenez pas à obtenir une présentation, le fichier a peut-être été supprimé par l’éditeur.

F I N

Plus de contenu associé

Transcription de la présentation

  1. Multimodal Emotion Recognition Colin Grubb Advisor: Nick Webb

  2. Motivation

  3. Multimodal fusion Research looking at audio, visual, and gesture information Feature Level vs. Decision Level Previous Research

  4. To what extent can we improve emotion recognition by using classification methods on audio and visual data? Research Question

  5. Set of rules vs. training a classifier Rule set is too basic Will use classifier to learn outputs of unimodal systems Decision Level Analysis

  6. EmoVoice (EMV) Real Time Audio Analysis Five emotional states w/ probabilities Published accuracy: 47.67% Audio System https://www.informatik.uni-augsburg.de/en/chairs/hcm/projects/emovoice/

  7. (Negative Active)  Angry (Negative Passive)  Sad (NEutral)  Neutral (PositveActive)  Happy (Positive Passive)  Content EmoVoice Confidence Levels negativeActive <0.40, 0.20, 0.10, 0.15, 0.15>

  8. Software created by Prof. Shane Cotter Uses still images Published accuracy: 93.4% Visual System

  9. Images Emotion: Happy Video Software Classifier Output: Happy I’m in a good mood! System Layout Emotion: Happy EmoVoice

  10. 8 subjects Five male, three female Audio Data Read sample sentences Visual Data Gather facial expressions from regular and long distance (6 ft.) Data Gathering

  11. 1 Weka Data Mining Software Used J48 Classifier C4.5 algorithm – decision tree Each branch represents decision made at that node 2 3 Experiments Output 1 Output 2 Output 3 Output 4 http://www.cs.waikato.ac.nz/ml/weka/

  12. Final dataset classifies between Happy Angry Neutral Sad Audio performance: 38.43% Visual performance: 77.43 % Emotion Classes

  13. Ran combined dataset against J.48 classifier Multimodal data initially ineffective Needed a way to improve dataset Initial Performance

  14. How can we use the two individual systems to complement each other? Two pieces of information: What does the visual system do poorly on? What kind of biases does EmoVoice have? Improving Accuracy

  15. Visual System Performs poorly at Neutral Some inaccuracy for all emotions tested EmoVoice Bias towards negative voice Very strong bias towards active voice Manual Bias

  16. Happy: For all happy training instances, if PP + PA > NA & NE & NP, change EMV Class to Happy Sad: If NP is 2nd to NA and within 0.05, change EMV Class to Sad Neutral: If NE tied with another confidence level, change EMV Class to Neutral If all probabilities within 0.05 of each other, change EMV Class to Neutral EmoVoice – Modification Rules

  17. Post Man. Bias Results

  18. Spring Practicum Refine rules Automation Online Classifier Mount on robot; cause apocalypse Future Work

  19. Questions? Comments? Thank you for listening.