1 / 61

Korean Brain Neuroinformatics Research Program and Artificial Auditory Systems

Korean Brain Neuroinformatics Research Program and Artificial Auditory Systems. Soo-Young Lee Brain Science Research Center Department of BioSystems Korea Advanced Institute of Science and Technology http:// bsrc.kaist.ac.kr. Korean Brain Neuroinformatics Research Program

nitesh
Télécharger la présentation

Korean Brain Neuroinformatics Research Program and Artificial Auditory Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. KoreanBrain Neuroinformatics Research Programand Artificial Auditory Systems Soo-Young Lee Brain Science Research Center Department of BioSystems Korea Advanced Institute of Science and Technology http://bsrc.kaist.ac.kr

  2. Korean Brain Neuroinformatics Research Program Artificial Auditory Systems- Auditory Model- Binaural Processing- Selective Attention Contents Department of BioSystems andBrain Science Research Center, KAIST

  3. Brain: The Final Frontier! Intelligence to Machine! Freedom to Mankind! From Biology to Hardware! Department of BioSystems andBrain Science Research Center, KAIST

  4. Feasibility study in early 1997 Announced by Ministry of Science and Technology on September 1997 Inauguration of Brain Science Research Center at Korea Advanced Institute of Science and Technology on December 1997 Brain Research Promotion Law on May 1998 Started on November 1998 for 10 years. Braintech’21 Department of BioSystems andBrain Science Research Center, KAIST

  5. 뇌기능의 응용 (신경회로망) Scope of Braintech’21 Brain Science & Engineering Research Program Neuroinformatics Application (Engineering) Macroscopic (Function) Under- standing (Science) Mimicking (Bioelectronics) Creating (Artificial Life) Neurobiology Protecting (Medical) Microscopic (molecules) Biomedical Brain Research Program Department of BioSystems andBrain Science Research Center, KAIST

  6. Phase 1 (1998-2000)- Build-up Foundation- 2 programs (Brain Science & Engineering, Brain Biomedical) Phase 2 (2001-2003)- Extend the Foundation- 3 programs (Brain Neurobiology, Brain Neuroinformatics, Brain Biomedical) Phase 3 (2004-2007)- Apply to Real-World Problems 3 Phases for 10 Years Department of BioSystems andBrain Science Research Center, KAIST

  7. Phase 1 VLSI Self-adapt Viosn & Auditory Cog. Sci. Modeling Phase 2 Auditory Vision Cog/Infer. Behavior fMRI Brain Eng. Cog. NS Phase 3 fMRI etc. Brain-like Systems (Artificial Brain) Phase 2: Neuroinformatics Program Department of BioSystems andBrain Science Research Center, KAIST

  8. Brain Neuroinformatics Brain-like Functional Systems Neuromorphic Chips Mathematical Model Analysis Software Neuroscience Data Functional Groups Measurement Technology Measurement Group Department of BioSystems andBrain Science Research Center, KAIST

  9. Selection and Concentration Integration- Database- Neuromorphic Chips- Brain-like Systems Multidisciplinary Team- Neuroscientists- Mathematicians- Engineers Korean Approaches Department of BioSystems andBrain Science Research Center, KAIST

  10. Pattern Recognition Cognition & Inference Vision Inference Brain-like System SelectiveAttention Learning/ Memory/ Language Behavior Recogn./ Tracking Feature Extraction Sen- sor Individual/ Group Behavior Perception/ Planning Body Control Feature Extraction Recog./ Understand Sen- sor Fusion Selective Attention Smell Olf. Touch Auditory Speech Recognition Brain Functions Department of BioSystems andBrain Science Research Center, KAIST

  11. Noise-Robust Speech Processing Cocktail party problem Artificial Auditory System Department of BioSystems andBrain Science Research Center, KAIST

  12. Auditory model • Binaural processing • Selective attention • Speech processing chip Artificial Auditory System • Based on Human Cognitive Mechanism • Develop mathematical model and auditory chip • Develop Continuous speech recognition system 95% recognition in 10dB SNR Department of BioSystems andBrain Science Research Center, KAIST

  13. Auditory Central Nerve System Department of BioSystems andBrain Science Research Center, KAIST

  14. Object Path Attention Spatial Path Cochlea Nonlinear Features Cross Correlation Bottom-Up Attention Time Adaptation Sound Localization TD/BU Attention Time-Freq. Masking Speech Enhancement Speaker Separation Complex Sound Top-Down Attention Aud. Cortex Higher-Order Functions Parser Speech Rec. System Research Scopes Auditory Pathway Department of BioSystems andBrain Science Research Center, KAIST

  15. Human Auditory System Department of BioSystems andBrain Science Research Center, KAIST

  16. Auditory Model - bandpass filtering (logarithmic scale in frequency) at basilar membrane - Nonlinear processing at inner hair cell (short-term adaptation, synchronization, etc.) Department of BioSystems andBrain Science Research Center, KAIST

  17. Auditory Model BM OHC IHC Synapse / Auditory Nerve Automatic Gain Control of Outer Hair Cell Department of BioSystems andBrain Science Research Center, KAIST

  18. Auditory model S Cochlear Filter 1 Auditory Nerve Fibers Gain & Q Control Frequency Information Cochlear Filter i Zero-Crossing Detector Time-Freq. Histogram x(t) y(t,f) Amplitude Integrator Nonlinear Transform Intensity Information Cochlear Filter M Auditory Nerve Fibers Basilar Membrane Frequency Selectivity Mechanical-to-neural Transduction Department of BioSystems andBrain Science Research Center, KAIST

  19. 100.0 90.0 80.0 Recognition Rates 70.0 60.0 50.0 40.0 30.0 20.0 10.0 0.0 Clean 25 20 15 10 5 SNR (dB) Recognition Results with Noise White Gaussian Noise Car Noise 10 5 0 -5 -10 SNR (dB) LPCC MFCC SBCOR PLP EIHC AudModel Department of BioSystems andBrain Science Research Center, KAIST

  20. 100.0 90.0 80.0 70.0 60.0 50.0 40.0 30.0 20.0 10.0 0.0 Recognition Results with Noise Factory Noise Military Operations Room Noise Recognition Rates Clean 25 20 15 10 5 Clean 25 20 15 10 5 SNR (dB) SNR (dB) LPCC MFCC SBCOR PLP EIHC AudModel Department of BioSystems andBrain Science Research Center, KAIST

  21. ICA-based Speech Features : Time Department of BioSystems andBrain Science Research Center, KAIST

  22. ICA-based Speech Features : Freq. Department of BioSystems andBrain Science Research Center, KAIST

  23. Time-Frequency Masking • Lateral Inhibition Department of BioSystems andBrain Science Research Center, KAIST

  24. Masking using Lateral Inhibition Department of BioSystems andBrain Science Research Center, KAIST

  25. Isolated Word Recognition Rates Department of BioSystems andBrain Science Research Center, KAIST

  26. Left Ear Z-1 Z-1 Z-1 Right Ear Z-1 Z-1 Z-1 Filter Banks (f) Interaural Time Difference (t) Binaural Hearing • Summation for all BP filters • Winner-Take-All on ITD Department of BioSystems andBrain Science Research Center, KAIST

  27. Audio Noise Mixed Signal Speech Signal Unknown Noise Mic 1 Command ANC + BSS Mixed Signal Audio Noise Mic 2 Noise Reference Signal Noise Car Audio Simultaneous ANC and BSS Department of BioSystems andBrain Science Research Center, KAIST

  28. Cochlea Cochlear Nucleus Auditory Cortex Higher-Order Brain Function (Recognition) Superior Olive  Superior Olive Auditory Cortex Cochlear Nucleus Cochlea Auditory Pathway •  Improvement of Cochlea model • Binaural hearing model • Top-Down attention • Speech-recognition chip Department of BioSystems andBrain Science Research Center, KAIST

  29. Unmixed Signal u1(t)= s1(t) Mixed Signal x1(t) Speech s1(t)=s(t) y1(t) + + A21 W21 W12 A12 Noise s2(t)=n(t) y2(t) Mixed Signal x2(t) Unmixed Signal u2(t)= s2(t) Blind Signal Separation Department of BioSystems andBrain Science Research Center, KAIST

  30. Speech+ Convolved Noise x(t)=s(t)+n0(t) Enhanced Speech u(t) Speech s(t) y(t) + -Wl Wl Noise Nl(t) Noise nl(t) + Active Noise Canceling Department of BioSystems andBrain Science Research Center, KAIST

  31. LMS: second-order statistics only ICA: statistical independence LMS algorithm SNR (dB) ICA-based algorithm Car Speech Music 45 40 35 30 25 20 15 10 5 0 Independent Component Analysis Department of BioSystems andBrain Science Research Center, KAIST

  32. 4 input buffer, 6 weight vector 12 KHz, 12 bit sampling 512 time delays Memory Requirement Input buffer : 2 x [ 2 x 512 x 12 ] : 12 MHz Weight vector : 2 x [ 3 x 512 x 30 ] : 36 MHz Function 4 ANC 2 BSS + 4 ANC 3 BSS Gate count 20,000 NESS processor Department of BioSystems andBrain Science Research Center, KAIST

  33. Artificial mixing using room response filter Simulation (1) Department of BioSystems andBrain Science Research Center, KAIST

  34. Simulation Tasks Simulation (2) Department of BioSystems andBrain Science Research Center, KAIST

  35. TASK II : 4 ch ANC M1 O1 S1 N1 N2 N3 N4 SNR Department of BioSystems andBrain Science Research Center, KAIST

  36. Simulation Result Department of BioSystems andBrain Science Research Center, KAIST

  37. Known Signal : 4 MUSICs Unknown Signal : CAR Demo System Department of BioSystems andBrain Science Research Center, KAIST

  38. 4 audio music 1 male voice 1 female voice MIC1 MIC2 OUT1 OUT2 2-BSS + 4-ANC Channels Department of BioSystems andBrain Science Research Center, KAIST

  39. Training TI DIGIT training set : men only 794 utterance Clean DB only ESR700 Development Tool Network size : automatically constructed by ESR700 tool Input : 256 Hidden : 20 Output : 10 Recognition Test (1) Department of BioSystems andBrain Science Research Center, KAIST

  40. Test TI DIGIT test set : men only 1267 utterance Artificial mixing by room response 2 BSS + 4 ANC BSS : Test DB + car noise [NOISEX CD] ANC : 4 MUSICs Recognition Test (2) Department of BioSystems andBrain Science Research Center, KAIST

  41. Case Example SNR : - 10 dB Recognition Test (3) MIC1 MIC2 OUT1 OUT2 Department of BioSystems andBrain Science Research Center, KAIST

  42. without ANC/BSS with ANC/BSS Recognition Rate(%) 100 80 60 40 20 0 Clean 25 20 15 10 5 0 -5 -10 SNR(dB) Recognition Rates Department of BioSystems andBrain Science Research Center, KAIST

  43. Recognition Test (4) Department of BioSystems andBrain Science Research Center, KAIST

  44. ICA-based ANC with Filter Bank Department of BioSystems andBrain Science Research Center, KAIST

  45. Filter bank design Eight-channel oversampled filter bank Prototype filter with 192 taps Learning curves Signal and noise: speech Mixing filter A measured filter in a normal office room with 1000 taps Performance limitation of the freq. domain approach The filter bank approach Much faster convergence speed than the time domain approach ANC Results withFilter Bank Department of BioSystems andBrain Science Research Center, KAIST

  46. Internal Cue External Cue Attended Output Classifier Output Bottom-Up Recognition Top-Down Attention Attended Features Bottom-Up Attention Input Features Brain Environment Input Stimulus Selective Attention • Bottom-Up: - Change - Masking - ICA • Top-Down: - MLP - HMM Department of BioSystems andBrain Science Research Center, KAIST

  47. Confusing/Occluded patterns Overlapped patterns Top-Down Selective Attention Department of BioSystems andBrain Science Research Center, KAIST

  48. 1958: Broadbent, “Early filter” theoryThe brain temporarily retained information about all stimuli but that the information faded, unless attention had been turned quickly to a particular memory trace. 1960: Treisman”the filter merely attenuates” 1971: Broadbent“selection on the basis of semantic properties” 1988: Fukushima: “Neocognitron” modelselective attention and switching for the recognition of superimposed binary patterns Background Department of BioSystems andBrain Science Research Center, KAIST

  49. S. Treue & J.C.M. Trujilo, Nature 1999, pp. 575-579. Feature Similarity Gain Model • The regulation of the gain of a sensory neuron reflects the similarity of the features of the currently behaviourally relevant target and the sensory selectivity of the neuron. • The relevant target features include the spatial location, direction of motion and presumably others. Department of BioSystems andBrain Science Research Center, KAIST

  50. ^ ^ ^ ^ h1 h2 h3 hK x1 x2 x3 xN a1 a2 a3 aN Wkn y1 y2 y3 yM Vmk x1 x2 x3 xN ^ x a x W h V y Proposed Neural Model • Early filtering: “feature-based”attention • Attention cue at test phase: external cue to a class • (focus at recognition only, not training) •  Top-Down expectation: Gradient-based EBP • algorithm using trained knowledge in an MLP Department of BioSystems andBrain Science Research Center, KAIST

More Related