Phonology and phonetics of tone perception

Phonology and phonetics of tone perception Deepti Ramadoss Luigi Burzio, Paul Smolensky, Colin Wilson IGERT Conference February 11-12, 2012

Introduction: • Speech perception • Tone languages , Tone perception • Experiments • Experiment 1 • Experiment 2 • Data Analysis Models • Theory and Grammar of tone perception • Conclusion

Introduction: Speech Perception Evidence of both phonological and phonetic knowledge used in perception Examples of some effects observed in speech perception studies: • Massaro and Cohen (1983): Perception makes use of phonotactic knowledge: • /l/ to /r/ continuum –preceded by consonant /t/ or /s/; since */tl/, */sr/: the boundary shifts to allow more legal clusters • Mann and Repp (1981): Perception expects coarticulation: • /t/ to /k/ continuum – preceded by consonant (/s/ or /S/) ;perception based on an expectation of coarticulation of [+/- anterior] feature Questions: • How can this be formally characterized? • What will this be like in the domain of tones?

Introduction: Tones and tone languages Languages that use pitch contrastively (contributes to word meaning) Syllables: consonants, vowels and tones e.g. Mandarin, Cantonese, Thai, Swedish, Yoruba, etc. • Thai • 5 tone system Falling /na:/ ‘face’ Rising /na:/ ‘thick’ High /na:/ ‘aunt’ Low /na:/ ‘custard apple’ Mid /na:/ ‘rice field’

Introduction: Tone perception • Tone perception cues: • Phonetic representation: entire f0 tone contour (Trajectory hypothesis; e.g. Xu 2004)

Introduction: Tone perception • Tone perception cues: • Phonetic representation: entire f0 tone contour (Trajectory hypothesis; e.g. Xu 2004) • Phonological representation: peaks/valleys high(H) and low(L) targets and their alignment (Target hypothesis; e.g. Morén and Zsiga 2006) Morén and Zsiga (2006) representation of Thai tone targets

Introduction: Tone perception • Tone perception cues: • Phonetic representation: entire f0 tone contour (Trajectory hypothesis; e.g. Xu 2004) • Phonological representation: peaks/valleys high(H) and low(L) targets and their alignment (Target hypothesis; e.g. Morén and Zsiga 2006) • Both phonetic and phonological (Hybrid hypothesis; current proposal)

Introduction: • Speech perception • Tone languages • Tone perception • Experiments • Experiment 1 • Experiment 2 • Data Analysis Models • Theory and Grammar of tone perception • Conclusion

Experiments to test the Hybrid hypothesis • Experiment 1: to test target alignment (phonology) • Manipulating target alignment by maintaining trajectories as naturally as possible • Experiment 2: to test trajectories (phonetics) • Manipulating trajectories by maintaining targets at their designated alignment • Targets and trajectories are inextricably intertwined – logically impossible to separate them to manipulate them • Instead each experiment accentuates the role of one or the other

Experiments to test the Hybrid hypothesis • Experiment 1: tests target alignment Manipulating tones with 2 targets: • Manipulating tones with 1 target:

Results - Synopsis • Results from most cases in this experiment show that the tone identified is consistent with target profiles for tone categories. • Rising tone manipulations may indicate that including knowledge of trajectories can better explain the data

Experiment 2: tests trajectory

Results: Synopsis • For all tone categories, for a majority of comparisons, there was a significant effect on categorization or goodness • Targets and alignments were left largely unaltered • Hence Target hypothesis cannot explain all these effects.

Summary • Experiment 1: Target knowledge • Experiment 2: Trajectory knowledge • Hybrid hypothesis: includes knowledge of both targets and trajectories. Hence, can account for the results from both these experiments

Data Analysis: Models • From the experiments: for each stimulus, an observed probability distribution can be calculated across all 5 tonal categories for each stimulus. is the observed probability of response k given stimulus j, weighted by its goodness.

Data Analysis: Models • Assume some distance metric between each original tone exemplar k {1, …, 5} and each manipulated stimulus j • Predicted probabilities - derived from Shepard’s Universal Law of Generalization (1987) (The probability that a response learned to any stimulus will generalize to any other is an invariant monotonic function of the distance between them. To a good approximation, this probability of generalization decays exponentially with the distance)

Data Analysis: Models • Predicted probabilities are computed based on a distance measure between each original tone exemplar and each stimulus • The nature of this distance measure can vary: the distance can be computed between • entire tone trajectories (Trajectory hypothesis) • targets at their aligned positions (Target hypothesis) • both tone trajectories and targets at their aligned positions (Hybrid hypothesis) • These distances can also be computed between the f0 values in the raw acoustic signals, or some transformation of these acoustic signals. The latter case assumes that the perceptual system performs this transformation before computing distances.

Data Analysis: Models Transformation of input: • Acoustic input • Derivative input • Parsons code input

Data Analysis: Models Results -Summary VE = 1

Introduction: • Speech perception • Tone languages • Tone perception • Experiments • Experiment 1 • Experiment 2 • Data Analysis Models • Models • Theory and Grammar of tone perception • Conclusion

Theory of perception From Shepard’s Universal Law of generalization: where, sum of the scaling parameters is defined as Then and

Grammar of tone perception • Probabilistic version of Harmonic Grammar based in Harmony theory (Smolensky 1986). • Constraints: • Harmony: sum of weighted constraint violations: • Probabilities can be assigned based on the Harmony computed per category

Grammar of tone perception • HG tableau for the original Falling tone

Conclusion • Speech perception makes direct use of both phonetic and phonological knowledge • Tone perception: trajectory and target knowledge play important roles in perception • Evidence from present work indicates listeners have knowledge of both trajectories and targets • Fitting models to experimental data shows that a model that makes use of a hybrid of trajectory and target knowledge best characterizes the data • The model with the best fit lends itself to a Harmonic Grammar analysis – thereby characterizing a grammar of tone perception • The perception grammar makes use of the same machinery that phonologists make use of in characterizing production – this machinery is hence not task dependent.

Thank you Luigi Burzio and Paul Smolensky Colin Wilson Lisa Zsiga SanjeevKhudanpur Vatsun Sadagopan Don Mathis Sharmi Seshamani Rattima Nitisaroj Thai Students Association at JHU Linguistics Lab at Georgetown University Members of the Phonetics Phonology lab CogSci, JHU

Thank you

Introduction: Speech Perception Evidence of both phonological and phonetic knowledge used in perception Examples of some effects observed in speech perception studies: • Massaro and Cohen (1983): Perception makes use of phonotactic knowledge: • /l/ to /r/ continuum –preceded by consonant /t/ or /s/; since */tl/, */sr/: the boundary shifts to allow more legal clusters • Mann and Repp (1981): Perception expects coarticulation: • /t/ to /k/ continuum – preceded by consonant (/s/ or /S/) ;perception based on an expectation of coarticulation of [+/- anterior] feature Questions: • How can this be formally characterized? • What will this be like in the domain of tones?

Introduction: Tones and tone languages Languages that use pitch contrastively (contributes to word meaning) Syllables: consonants, vowels and tones e.g. Mandarin, Cantonese, Thai, Swedish, Yoruba, etc. • Thai • 5 tone system Falling /na:/ ‘face’ Rising /na:/ ‘thick’ High /na:/ ‘aunt’ Low /na:/ ‘custard apple’ Mid /na:/ ‘rice field’

Introduction: Tone perception • Tone perception cues: • Phonetic representation: entire f0 tone contour (Trajectory hypothesis; e.g. Xu 2004)

Introduction: Tone perception • Tone perception cues: • Phonetic representation: entire f0 tone contour (Trajectory hypothesis; e.g. Xu 2004) • Phonological representation: peaks/valleys high(H) and low(L) targets and their alignment (Target hypothesis; e.g. Morén and Zsiga 2006) Morén and Zsiga (2006) representation of Thai tone targets

Introduction: Tone perception • Tone perception cues: • Phonetic representation: entire f0 tone contour (Trajectory hypothesis; e.g. Xu 2004) • Phonological representation: peaks/valleys high(H) and low(L) targets and their alignment (Target hypothesis; e.g. Morén and Zsiga 2006) • Both phonetic and phonological (Hybrid hypothesis; current proposal)

Experiments to test the Hybrid hypothesis • Experiment 1: to test target alignment (phonology) • Manipulating target alignment by maintaining trajectories as naturally as possible • Experiment 2: to test trajectories (phonetics) • Manipulating trajectories by maintaining targets at their designated alignment • Targets and trajectories are inextricably intertwined – logically impossible to separate them to manipulate them • Instead each experiment accentuates the role of one or the other

Experiments to test the Hybrid hypothesis • Experiment 1: tests target alignment Manipulating tones with 2 targets: • Manipulating tones with 1 target:

Behavioral Experiment • 10 native speakers of Thai • 3 blocks – 1 repetition per block • Participants were asked to listen to a stimulus • They were then presented with 5 choices : • The target items varied only by tone • After making a choice, they were asked to give a ‘goodness’ rating • Results from most cases in this experiment show that the tone identified is consistent with target profiles for tone categories. • Hence, data cannot show conclusively that knowledge of target alignment alone is insufficient. • One case may indicate that including knowledge of trajectories can better explain the data

Results Rising Manipulations: Choices More ‘High’ judgments when the first target is aligned early in the syllable

Results Rising Manipulations • Rising manipulation often classified as a High tone • Target hypothesis cannot explain this if f0 values in the relevant windows in R’ are similar to R targets • R’ similar to H in shape • R and H confusability may also arise from shape similarity

Experiment 2: tests trajectory Form of manipulation: v = original trajectory u = straight line trajectory v + (1–)u –2<–1  < 1<2 u; v;

Experiment 2: tests trajectory

Results: Falling manipulations Significantly affects goodness rating of Falling manipulations computed using a mixed effects model

Results: Rising manipulations Significantly affects categorization of Rising manipulations computed using a mixed effects model

Results: Synopsis • For all tone categories, for at least 3 comparisons, there was a significant effect on categorization or goodness • Targets and alignments were left largely unaltered • Hence Target hypothesis cannot explain all these effects.

Summary • Experiment 1: Target knowledge • Experiment 2: Trajectory knowledge • Hybrid hypothesis: includes knowledge of both targets and trajectories. Hence, can account for the results from both these experiments

Data Analysis: Models • From the experiments: for each stimulus, an observed probability distribution can be calculated across all 5 tonal categories for each stimulus. is the observed probability of response k given stimulus j, weighted by its goodness.

Data Analysis: Models • Assume some distance metric between each original tone exemplar k {1, …, 5} and each manipulated stimulus j • Predicted probabilities - derived from Shepard’s Universal Law of Generalization (1987) (The probability that a response learned to any stimulus will generalize to any other is an invariant monotonic function of the distance between them. To a good approximation, this probability of generalization decays exponentially with the distance)

Phonology and phonetics of tone perception