1 / 81

Hallucinations in Auditory Perception!!!

Hallucinations in Auditory Perception!!!. Malcolm Slaney Yahoo! Research Stanford CCRMA. Hadoop. One Dimensional (waveform). Pressure. Time. Cochlear Processing. Two Dimensional (not a spectrogram). Cochlear. Place. Time. Correlogram Processing. Three Dimensional (neural movie).

aleon
Télécharger la présentation

Hallucinations in Auditory Perception!!!

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Hallucinations in Auditory Perception!!! Malcolm Slaney Yahoo! Research Stanford CCRMA

  2. Hadoop

  3. One Dimensional (waveform) Pressure Time Cochlear Processing Two Dimensional (not a spectrogram) Cochlear Place Time Correlogram Processing Three Dimensional (neural movie) Cochlear Place Time Autocorrelation Lag

  4. Correlogram Distance down cochlea Center Frequency Time Interval (s) Autocorrelation Lag With help from Richard O. Duda

  5. Correlogram

  6. Success Reconstructing from correlogram NIPS Keynote

  7. Continuation Tone and Noise Parliament Cough Hear two voices? What do you hear? Waveforms? Ideas? Problems

  8. Pressure Time Cochlear Processing Cochlear Place Time Correlogram Processing Cochlear Place Time Autocorrelation Lag

  9. Speech Examples Wedding Sine Natural

  10. What Vowel is This? Word 1 Word 2 Peter Ladefoged Word 3

  11. McGurk

  12. Speech Vision Speech Audio Locate Environment Audio Locate Vision Object Speech Speech Object Vision Wedding Vowel? Ventroloquism Dots McGurk Sinewave

  13. ASR Three Three Three Language model for the words: “one”, “two”, “three” Two Two Two One One One Word model showing phonemes for the word one /w/ / / /n/ Acoustic (phoneme) model for the phoneme / / S1 S2 S3

  14. Conventional Scene Analysis Slide by Dan Ellis (Columbia)

  15. Barker—ASR

  16. Goto—CASA with MIDI MIDI Sequence

  17. Old plus New Principle Slide by Dan Ellis (Columbia)

  18. Ellis—Prediction Driven

  19. Saliency

  20. Saliency Example • Time-frequency display • Saliency map shows high-interest locations

  21. Saliency Maps • Longer tones better • Missing parts salient • Modulation more salient • Forward masking works

  22. Sound Examples • Birds • Calls • Cows • Horse • Waterfall

  23. Saliency Comparison • Details of saliency comparison • Model predictions

  24. X Z M Z M M X Y Y m M Relational Network (Simple) • Patches of neurons • Each measureone quantity • Bidirectionalrelations for feedback/feedforward Thanks to Rodney Douglas

  25. Relational specification Input here Relational feedback RelationalFeedback Relational Network (example)

  26. ASR Relational Network Bidirectional links enforce phoneme/word constraints Phone Recognizer Cochlea Word Recognizer Phone Recognizer Delay A patch of neurons (one of N output) Note: We don’t know how to represent delays

  27. Without A A A I Desired Results Relational Feedback With /A/ Phoneme Patch /I/ Phoneme Patch AI Word Patch IA Word Patch Phoneme Input

  28. Simulation

  29. Simulation 2

  30. Simulation 3

  31. Grossberg—ART

  32. ICA Different distributions One Microphone GMM models of distribution Statistical Means

  33. Conventional

  34. Better?

  35. Thanks malcolm@ieee.org

  36. Pitch

  37. Silicon Frequency Response • Tone ramps into two cochleas

  38. Cochlear Best Frequency

  39. Cochlear Rate Profiles Spikes per utterance Left Cochlea Right Cochlea

  40. Hardware Overview Phoneme Word Cochlea Learning PCI-AER (for remapping) Learning Cochlea Learning Giacomo Indiveri Shih-Chii Liu PCI-AER (for remapping) Implemented in MATLAB

  41. LSH Movie

More Related