1.38k likes | 1.48k Vues
Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon. Bob McMurray University of Iowa Dept. of Psychology. Peter Ladefoged (1925 – 2006). He taught us all phonetics. Collaborators. Richard Aslin Michael Tanenhaus David Gow. Joe Toscano
E N D
Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon. Bob McMurray University of Iowa Dept. of Psychology
Peter Ladefoged (1925 – 2006) He taught us all phonetics
Collaborators Richard Aslin Michael Tanenhaus David Gow Joe Toscano Cheyenne Munson Meghan Clayards Dana Subik The students of the MACLab
In language, information arrives sequentially. • Partial syntactic and semantic representations are formed as words arrive. The cowboys chased the linguists away… • Words are identified over • sequential phonemes.
Spoken Word Recognitionis an ideal arena in which to study these issues because: • Speech production gives us a lot of rich temporal information to use in this way. • We have a clear understanding of the input (from phonetics). • The output is easy to measure online
Online Comprehension • Listeners form hypotheses as the input unfolds. • Need measurements of how listeners interpret speech, moment-by-moment. • May reveal how information is integrated: • Discreteness vs. Gradiency • Combinatorial Units
Mechanisms of Temporal Integration • Stimuli do not change arbitrarily. • Perceptual cues reveal something about the change itself. • Active integration: • Anticipating future events • Retain partial present representations. • Resolve prior ambiguity.
Overview • Speech perception and Spoken Word Recognition. 2) Lexical activation is sensitive to fine-grained detail in speech. 3) Where we’re going, we don’t need phonemes: evidence for continuous information integration. 4) Back in time: staying off the garden-path. 5) Forward to the future: coping with (and benefiting from) with phonological modification.
X basic bakery bakery X ba… kery barrier X X bait barricade X baby • Online Word Recognition • Information arrives sequentially • At early points in time, signal is temporarily ambiguous. • Later arriving information disambiguates the word.
Current models of spoken word recognition • Immediacy: Hypotheses formed from the earliest moments of input. • Activation Based: Lexical candidates (words) receive activation to the degree they match the input. • Parallel Processing: Multiple items are active in parallel. • Competition: Items compete with each other for recognition.
Input: b... u… tt… e… r time beach butter bump putter dog
These processes have been well defined for a phonemic representation of the input. l gw d But considerably less ambiguity if we consider subphonemic information. Example: subphonemic effects of motor processes.
Coarticulation n n ee t c k Any action reflects future actions as it unfolds. Example:Coarticulation Articulation (lips, tongue…) reflectscurrent, futureandpastevents. Subtle subphonemic variation in speech reflects temporal organization. Sensitivity to theseperceptualdetails might yield earlier disambiguation.
These processes have largely been ignored because of a history of evidence that perceptual variability gets discarded. Example:Categorical Perception
Categorical Perception B 100 100 Discrimination % /p/ Discrimination ID (%/pa/) 0 0 B VOT P • Sharp identification of tokens on a continuum. P • Discrimination poor within a phonetic category. Subphonemic variation in VOT is discarded in favor of adiscretesymbol (phoneme).
Sense Categorical Perception (CP) • Defined fundamental computational problems. • CP is output of • Speech perception • Input to • Phonology • Word recognition. Words Phonology Phonemes Sound
Evidence against the strong form of Categorical Perception from psychophysical-type tasks: • Discrimination Tasks • Pisoni and Tash (1974) • Pisoni & Lazarus (1974) • Carney, Widin & Viemeister (1977) Classic explanation: Auditory tasks: non-categorical Phonological tasks: categorical Paradigmatic CP: within-category variation is noise. Not important to higher language. • Training • Samuel (1977) • Pisoni, Aslin, Perey & Hennessy (1982) • Goodness Ratings • Miller (1997) • Massaro & Cohen (1983)
Sense Categorical Perception (CP) Enables a divide-and-conquer approach. Words Cont. cues (non-CP) Phonology Phonemes Sound
Sense Categorical Perception (CP) Enables a divide-and-conquer approach. Words • But, assumes that • Speech tasks tap phonemes (or something like them) • Phonemes (or something like them) are legitimate processing units. Phonology Phonemes Sound
Minimal computational problem: Computing meaning. Words Phonology Phonemes Sound
? CP Minimal computational problem: Computing meaning. CP tasks don’t necessarily tap a stage of this problem. Words Phonology Phonemes Sound
Minimal computational problem: Computing meaning. CP tasks don’t necessarily tap a stage of this problem. Words Phonology Phonemes Lexical representation: clearly a component. Sound Goal: Reassess continuous sensitivity (non-CP) w.r.t. words
Experiment 1 ? Does within-category acoustic detail systematically affect higher level language? Is there a gradient effect of continusuous acoustic detail on lexical activation?
Experiment 1 A gradient relationshipwould yield systematic effects of subphonemic information on lexical activation. If this gradiency is useful for temporal integration, it must be preserved over time. Need a design sensitive to bothacoustic detailand detailedtemporal dynamicsof lexical activation. McMurray, Aslin & Tanenhaus (2002)
Acoustic Detail Use a speech continuum—more steps yields a better picture acoustic mapping. KlattWorks:generate synthetic continua from natural speech. • 9-step VOT continua (0-40 ms) • 6 pairs of words. • beach/peach bale/pale bear/pear • bump/pump bomb/palm butter/putter • 6 fillers. • lamp leg lock ladder lip leaf • shark shell shoe ship sheep shirt
Temporal Dynamics How do we tap on-line recognition? With an on-line task:Eye-movements Subjects hear spoken language and manipulate objects in a visual world. Visual world includes set of objects with interesting linguistic properties. abeach, apeachand some unrelated items. Eye-movements to each object are monitored throughout the task. Tanenhaus, Spivey-Knowlton, Eberhart & Sedivy, 1995
Why use eye-movements and visual world paradigm? • Relatively naturaltask. • Eye-movements generated veryfast(within 200ms of first bit of information). • Eye movementstime-lockedto speech. • Subjectsaren’t awareof eye-movements. • Fixation probability maps ontolexical activation..
Task A moment to view the items
Task Bear Repeat 1080 times
Identification Results 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 5 10 15 20 25 30 35 40 High agreement across subjects and items for category boundary. proportion /p/ B VOT (ms) P By subject:17.25 +/- 1.33ms By item: 17.24 +/- 1.24ms
Task 200 ms Trials 1 2 3 4 5 % fixations Time Target = Bear Competitor = Pear Unrelated = Lamp, Ship
Task 0.9 VOT=0 Response= VOT=40 Response= 0.8 0.7 0.6 0.5 Fixation proportion 0.4 0.3 0.2 0.1 0 0 400 800 1200 1600 2000 0 400 800 1200 1600 Time (ms) More looks to competitor than unrelated items.
Task target Fixation proportion Fixation proportion time time • Given that • the subject heard bear • clicked on “bear”… How often was the subject looking at the “pear”? Categorical Results Gradient Effect target target competitor competitor competitor competitor
Results 20 ms 25 ms 30 ms 10 ms 15 ms 35 ms 40 ms 0.16 0.14 0.12 0.1 0.08 0.06 0.04 0.02 0 0 400 800 1200 1600 0 400 800 1200 1600 2000 Response= Response= VOT VOT 0 ms 5 ms Competitor Fixations Time since word onset (ms) Long-lasting gradient effect: seen throughout the timecourse of processing.
0.08 0.07 0.06 0.05 0.04 0.03 0.02 0 5 10 15 20 25 30 35 40 Area under the curve: Clear effects of VOT B: p=.017* P: p<.001*** Linear Trend B: p=.023* P: p=.002*** Response= Response= Looks to Competitor Fixations Looks to Category Boundary VOT (ms)
0.08 0.07 0.06 0.05 0.04 0.03 0.02 0 5 10 15 20 25 30 35 40 Unambiguous Stimuli Only Clear effects of VOT B: p=.014* P: p=.001*** Linear Trend B: p=.009** P: p=.007** Response= Response= Looks to Competitor Fixations Looks to Category Boundary VOT (ms)
Summary Subphonemic acoustic differences in VOT have gradient effect on lexical activation. • Gradient effect of VOT on looks to the competitor. • Effect holds even for unambiguous stimuli. • Seems to be long-lasting. Consistent with growing body of work using priming (Andruski, Blumstein & Burton, 1994; Utman, Blumstein & Burton, 2000; Gow, 2001, 2002).
An alternative framework Word recognition is systematically sensitive to subphonemic acoustic detail. 2) Continuous acoustic detail is represented as gradations in activation across the lexicon. 3) This can do the work of sublexical units like phonemes. 4) Gradient sensitivity coupled to normal word recognition processes enables the system to take advantage of subphonemic regularities for temporal integration.
Lexical Sensitivity Word recognition is systematically sensitive to subphonemic acoustic detail. • Voicing • Laterality, Manner, Place • Natural Speech • Vowel Quality • Infant voicing categories
Extensions B Sh L P Word recognition is systematically sensitive to subphonemic acoustic detail. • Voicing • Laterality, Manner, Place • Natural Speech • Vowel Quality • Infant voicing categories Metalinguistic Tasks
Extensions 0.1 Response=P Looks to B 0.08 0.06 Competitor Fixations Response=B Looks to B 0.04 Category Boundary 0.02 0 0 5 10 15 20 25 30 35 40 VOT (ms) Word recognition is systematically sensitive to subphonemic acoustic detail. • Voicing • Laterality, Manner, Place • Natural Speech • Vowel Quality • Infant voicing categories Metalinguistic Tasks
Extensions 0.1 Response=P Looks to B 0.08 0.06 Competitor Fixations Response=B Looks to B 0.04 Category Boundary 0.02 0 0 5 10 15 20 25 30 35 40 VOT (ms) Word recognition is systematically sensitive to subphonemic acoustic detail. • Voicing • Laterality, Manner, Place • Natural Speech • Vowel Quality • Infant voicing categories Metalinguistic Tasks
Lexical Sensitivity Word recognition is systematically sensitive to subphonemic acoustic detail. • Voicing • Laterality, Manner, Place • Natural Speech • Vowel Quality • Infant voicing categories Metalinguistic Tasks ? Non minimal pairs • ? Duration of effect (Exp 3-4)
bump pump dump bun bumper bomb 2) Continuous acoustic detail is represented as gradations in activation across the lexicon. Input: b... u… m… p… time
3) This can do the work of sublexical units like phonemes. If lexical processes can represent speech detail, do we need sublexical processes? Perhaps: How are multiple cues (to the same phoneme) integrated? (Exp 2)
Temporal Integration 4) Gradient sensitivity coupled to normal word recognition processes enables the system to take advantage of subphonemic regularities for temporal integration. • Regressive ambiguity resolution (exp 3-5): • Ambiguity retained until more information arrives. • Progressive expectation building (exp 5-6): • Phonetic distinctions are spread over time • Anticipate upcoming material.
Overview • Speech perception and Spoken Word Recognition. 2) Lexical activation is sensitive to fine-grained detail in speech. 3) Where we’re going, we don’t need phonemes: evidence for continuous information integration. 4) Back in time: staying off the garden-path. 5) Forward in time: coping with (and benefiting from) with phonological modification.
words phonemes Substitute your favorite sublexical unit here (syllables, diphones, etc)… Traditional speech chain: signal-> phonemes -> words