Study of Neural Correlates of Mandarin Tonal Production with Neural Network Model

1. Study of Neural Correlates of Mandarin Tonal Production with Neural Network Model Chao-Min Wu* Tao-Wei Wang Department of Electrical Engineering, National Central University, Jhongli 320, Taiwan, ROC Received 6 May 2011; Accepted 15 July 2011; doi: 10.5405/jmbe.934 Chairman:Hung-Chi YangPresenter: Yue-Fong Guo Advisor: Dr. Yeou-Jiunn ChenDate: 2012.12.26

2. Outline • Introduction • Overview of the DIVA model • Method • Results • Discussion • Conclusion

3. Introduction • Population ages • Communicative disorders • Improve the diagnosis and treatment of speech problems • Analyzing Mandarin speech production • Neural correlates of Mandarin tonal production

4. Introduction • Physiological model • To Examine speech articulation • A neural-network-based direction into DIVA models （Directions Into Velocities Articulator ） • Simulate neural correlates of speech production • To determine the neural correlates of Mandarin tonal production

5. Introduction • The DIVA model • Originally design • Fixed preset pitch • Tonal production was not considered • The modified DIVA model • Produce four types of Mandarin tones • Simulate brain activity regions

6. Overview of the DIVA model • Adaptive neural network model • The steps • A word or a syllable input • Generate articulatory movement commands • Shape of the vocal tract required to produce • Speech synthesizer produce speech sound • Produces a sequence of number • Represent the brain activity regions and levels

7. Overview of the DIVA model

8. Method • DIVA model • Mandarin tonal production is described and incorporated • Simulating brain activity are presented

9. Method • Mandarin tonal production • Chao proposed a five-point-scale

10. Method • Pitch scaling function • Approximated with the fourth-order polynomials • Multiplied by the pitch periods of the first tone

11. Method • To modify the motor commands • Corresponding articulator to generate • The tonal speech • The corresponding brain activity regions

12. Method • Simulation of brain activity • Baseline condition • Speaking condition • Corresponding brain areas

13. Method • In the first simulation • Given vowel with different tones(/a/, /á/, /ã/, and /à/) • Were analyzed • Verify whether capable of tone production • In the second simulation • The brain activity regions of two different vowels with a given tone(/a/-/u/) • The difference of the corresponding vowel brain activity regions • Determine whether the original function maintained

14. Method • In the third simulation • Give vowel with different tones • A comparison among brain activity regions

15. Results • In the first experiment • According to the first two formant frequencies • The produced f0 contours • Capable of Mandarin tonal production

16. Results • In the second experiment • motor cortex, pre-motor cortex, auditory cortex, SMA, and cerebellum • (a) the vowel /a/ and (b) the vowel /u/

17. Results • Difference between the production of different vowels • Difference in the activation in the lip and laryngeal areas and the SSM area

18. Results • In the third experiment • Difference in the larynx area and the somatosensory cortex

19. Discussion • Previous study find the tone-relate activity regions • Prefrontal cortex • Pitch judgment tasks • Left middle temporal • Word-level comprehension • Right inferior frontal gyri • Precentralgyri

20. Discussion • The DIVA model simulating brain activity • Motor cortex • Auditory • Somatosensory • Cerebellum • The differences between simulation and study

21. Conclusion • Because these regions are not included in the DIVA model. • Future study • Focus on the tone-related brain region • Needed to include these region in the DIVA model • To investigate the role • The frontal and temporal lobes

22. The end