With thanks to: Dana Subik

1. Bob McMurray Bob McMurray Michael K. Tanenhaus Mickey K. TanenMouse Richard N. Aslin Richard N. Aslin University of Rochester With thanks to: Dana Subik Michael J. Spivey Cornell University The lexical/phonetic interface: Evidence for gradient effects of within-category VOT on lexical access

2. Prosody and consonants • We know prosodic domain has a large effect on vowels. • Recent evidence suggests it affects consonants, too. • Strong position in a prosodic domain characterized by: • Longer VOTs • Hyper-articulation (more extreme formant • transitions) • Burst amplitude

3. Prosody and consonants • Sensitivity to this information would: • help listeners recognize consonants in the face of contextual variation. • cue upcoming prosodic effects

4. Speech Perception Speech perception shows probabilistic effects of many information sources: Lexical ContextSpectral vs. Temporal Cues Visual Information Transition Statistics Speech Rate Stimulus Naturalness Sentential Context Compensatory Coarticulation Embeddings Syllabic Stress Lexical Stress Phrasal Stress A system that was sensitive to fine-grained acoustic detail would be much more efficient than one that did not.

5. B Discrimination ID (%/pa/) P • Sharp identification of speech sounds on a continuum • Discrimination poor within a phonetic category Categorical Perception CP suggests listeners are NOT sensitive to these differences. 100 % /p/ 0 B VOT P

6. Revisiting Categorical Perception? Some evidence against CP from Discrimination Tasks (Pisoni and Tash, 1974) Goodness Ratings (Miller, 1997) Discrimination Training (Samuel, 1977) Semantic Priming (Andruski, Blumstein & Burton, 1994) Very little evidence from ID tasks… Very little evidence for a gradient response… Perhaps a more sensitive measure?

7. Experiment 1: Categorical Perception 9-step /ba/ - /pa/ VOT continuum (0-40ms) Identification indicated by mouse click. Eye movements monitored at 250 hz. 17 Subjects

8. Experiment 1: Categorical Perception B P Ba 1 2 3

9. Experiment 1: Identification Results 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 5 10 15 20 25 30 35 40 Category Boundary 17.5 +/- .83ms Proportion of /p/ response B P VOT (ms) Steep ID function characteristic of Categorical Perception.Stimuli are good.

10. Experiment 1: Data Analysis • Analyze “competitor” effects: • E.g. Given that • the subject heard /ba/ • clicked on “ba”… • How often was the • Subject looking at • “pa”? Target (ba) Fixation proportion Competitor (pa) time

11. Experiment 1: Data Analysis Effective ID Function Actual ID Function 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 5 10 15 20 25 30 35 40 Proportion of /p/ response VOT (ms) Trials with low-frequency response excluded. Effectively yields a “perfect” categorization function.

12. Experiment 1: Eye movement data 0.9 0.8 0.7 B 0.6 P 0.5 P (VOT=15) 0.4 B (VOT=20) 0.3 0.2 0.1 0 0 400 800 1200 1600 2000 0 400 800 1200 1600 More fixations to competitor near the category boundary. VOT=0 Response=B VOT=40 Response=P Fixation proportion Time (ms) Fixations to competitor even on “endpoint” trials.

13. Experiment 1: Results and Conclusions • Steep slope for mouse response curves. • consistent with categorical perception • Small difference between stimuli near category boundary and others. • Consistent with previous research.

14. Experiment 1: However… • We are really interested in lexical activation… • This sort of task purports to measure phoneme (not lexical) activation • 2AFC tasks require metalinguistic judgments • What exactly are we measuring??? • 2AFC metalinguistic tasks may underestimate sensitivity to subphonemic acoustic information

15. Lexical sensitivity to subphonemic variation • Why would lexical sensitivity to subphonemic differences be a good idea? • Extract more information from the signal • Could be used to help resolve temporary phonetic/lexical ambiguities when subsequent information arrives (e.g. vowel length or sentential context)

16. Experiment 2: Lexical Identification Six 9-step /ba/ - /pa/ VOT continuum (0-40ms) Bear/Pear Beach/Peach Butter/Putter Bale/Pale Bump/Pump Bomb/Palm 12 L- and Sh- Filler items Leaf Lamp Ladder Lock Lip Leg Shark Ship Shirt Shoe Shell Sheep Identification indicated by mouse click on picture Eye movements monitored at 250 hz 17 Subjects

17. Experiment 2: Lexical Identification A moment to view the items

18. Experiment 2: Lexical Identification 500 ms later

19. Experiment 2: Lexical Identification Bomb

20. Experiment 2: Identification Results 1 0.9 Word function not as steep. 0.8 Exp 1: BP 0.7 Exp 2: Words 0.6 0.5 Category boundaries are the same. 0.4 0.3 0.2 0.1 0 0 5 10 15 20 25 30 35 40 BP: 17.5 +/- .83ms Wordssubject:17.25 +/-1.33ms Wordsitem: 17.24 +/- 1.24ms Boundaries proportion /p/ B VOT (ms) P

21. Experiment 2: Identification Results 1 0.9 0.8 0.7 0.6 Yields a “perfect” categorization function. 0.5 0.4 0.3 0.2 0.1 0 0 5 10 15 20 25 30 35 40 ID Function after filtering Actual Exp2 Data Again: Trials with low-frequency response excluded. proportion /p/ B VOT (ms) P

22. 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 400 800 1200 1600 2000 0 400 800 1200 1600 Experiment 2: Eye Movement Results VOT=0 Response= VOT=40 Response= Fixation proportion Time (ms) More looks to competitor than unrelated items

23. Experiment 2: Data Analysis • Gradient “competitor” effects: • E.g. Given that • the subject heard bomb • clicked on “bomb”… How often was the Subject looking at the “palm”? Categorical Results Gradient Effect target target Fixation proportion Fixation proportion competitor competitor time time

24. Experiment 2: Gradiency? 0.08 0.07 0.06 0.05 Andruski et al (schematic) Gradient Sensitivity 0.04 “Categorical” Perception 0.03 0.02 0 5 10 15 20 25 30 35 40 Looks to Looks to Fixation proportion VOT (ms)

25. 20 ms 25 ms 30 ms 10 ms 15 ms 35 ms 40 ms 0.16 0.14 0.12 0.1 0.08 0.06 0.04 0.02 0 0 400 800 1200 1600 0 400 800 1200 1600 2000 Experiment 2: Eye Movement Results Gradient effects of VOT? Response= Response= VOT VOT 0 ms 5 ms Fixation proportion Time since word onset (ms) Smaller effect on the amplitude of activation—more effect on the duration: Competitors stay active longer as VOT approaches the category boundary.

26. 0.08 0.07 0.06 0.05 0.04 0.03 0.02 0 5 10 15 20 25 30 35 40 Clear effects of VOT B: p=.017* P: p<.0001*** Linear Trend B: p=.023* P: p=.002** Experiment 2: Eye Movement Results Response= Response= Looks to Fixation proportion Looks to Category Boundary VOT (ms)

27. 0.08 0.07 0.06 0.05 0.04 0.03 0.02 0 5 10 15 20 25 30 35 40 Clear effects of VOT B: p=.017* P: p<.0001*** Linear Trend B: p=.023* P: p=.002** Experiment 2: Eye Movement Results Response= Response= Looks to Fixation proportion Looks to Category Boundary VOT (ms) Unambiguous Stimuli Only

28. 0.08 0.07 0.06 0.05 0.04 0.03 0.02 0 5 10 15 20 25 30 35 40 Experiment 2: Eye Movement Results Response= Response= Looks to Fixation proportion Looks to Category Boundary VOT (ms) • Replicates and extends Andruski et al (1994). • They compared stimuli near boundary to distant stimuli. • We demonstrate gradiency in between.

29. Experiment 2: Effect of Time? • How long does the gradient sensitivity to VOT remain? • Need to examine: • the effect of time on competitor fixations • interaction with VOT

30. Experiment 2: Effect of Time? Trial 1 Trial 7 Trial 3 Trial 8 Trial 2 Trial 5 Trial 6 Trial 4 early late • For each group, fixations from • only 1 time-bin were used • Early: 300-1100ms • Late: 1100-1900ms Early Late Analysis: • Randomly sorted trials into two groups (early and late). • Ensures independence of data in each time-bin (since each trial only contributes to one)

31. Experiment 2: Eye Movement Results 0.11 0.1 0.09 Early (300-1100ms) 0.08 0.07 Late (1100-1900ms) 0.06 0.05 0.04 0.03 0.02 0.01 0 5 10 15 20 25 30 35 40 Response= Response= Fixation proportion Looks to Looks to Category Boundary VOT (ms) Main effect of time /b/: p=.001*** /p/: p=.0001**** Main effect of VOT /b/: p=.015* /p/: p=.001*** Linear Trend for VOT /b/: p=.022* /p/: p=.009** No Interaction p>.1

32. Experiment 2: Eye Movement Results 0.11 0.1 0.09 Early (300-1100ms) 0.08 0.07 Late (1100-1900ms) 0.06 0.05 0.04 0.03 0.02 0.01 0 5 10 15 20 25 30 35 40 Response= Response= Fixation proportion Looks to Looks to Category Boundary VOT (ms) Main effect of time /b/: p=.001*** /p/: p=.0001**** Main effect of VOT /b/: p=.006** /p/: p=.013* Linear Trend for VOT /b/: p=.0012** /p/: p=.02** No Interaction p>.1

33. The ambiguous first consonant of uny is clearly a /k/ after hearing ”uny” g k Experiment 2: Temporal ambiguity resolution The lexical/phonetic identity of a segment can be determined by acoustic features that arrive after the segment in question. Thus, like in higher level language comprehension, temporal ambiguity resolution is an important issue.

34. Experiment 2: Temporal ambiguity resolution • Lexical/Phonetic Temporal Ambiguity can be caused by • Vowel length (cue to speaking rate and stress) • Lexical/Statistical effects • Embedded words • Subphonemic sensitivity can minimize or eliminate the effects of temporary phonetic ambiguity by • Storing how ambiguous a segment is • Keeping competitors active until resolution occurs.

35. Results and Conclusions Slope of identification curve varies as a function of task—classic 2AFC phoneme ID judgments underestimate subphonemic sensitivity. Subphonemic acoustic differences in VOT affect lexical activation. • Gradient effect of VOT on looks to the competitor • Effect holds even for unambiguous stimuli. • Effect is long-lasting. • VOT affects duration of activation, not amplitude. • Much smaller effect in non-lexical tasks (BP)

36. Results and Conclusions • Subphonemic effects on lexical activation seem consistent with a probabilistic parallel processing mechanism. • Gradient sensitivity to VOT could be used for • Early detection of prosodic domain • Resolving temporal phonetic ambiguities

37. Results and Conclusions Subphonemic variation in VOT is not discarded It is not butsignal. Lexical activation exhibits gradient effects of subphonemic (VOT) variation.

38. The lexical/phonetic interface: Evidence for gradient effects of within-category VOT on lexical access Bob McMurray Michael K. Tanenhaus Richard N. Aslin University of Rochester With thanks to: Dana Subik Michael J. Spivey Cornell University

39. Experiment 1: Eye movement data 0.3 0.25 15 ms 0 ms 25 ms 5 ms 0.2 20 ms 10 ms 30 ms 35 ms 0.15 40 ms 0.1 0.05 0 400 800 1200 1600 2000 0 0 400 800 1200 1600 Experiment 1 does show hints of gradiency VOT VOT Fixation proportion Time (ms) • Very small • Difference between stimuli near boundary and endpoints. • Maybe something gradient for /pa/.

40. Experiment 1: A BaPa Reprise 0.08 0.07 0.06 0.05 0.04 0.03 0.02 0.01 0 0 5 10 15 20 25 30 35 40 Hints of gradiency: /b/: p =.044 * /p/: p<.001 *** Could be driven by big differences near category boundary. (Consistent with Andruski et al) Response=B Looks to P Response=B Looks to P Fixation proportion Category Boundary VOT (ms)

41. Experiment 1: Eye movement data Experiment 1: Eye movement data 0.08 0.07 0.06 0.05 0.04 0.03 0.02 0.01 0 0 5 10 15 20 25 30 35 40 Remove items near boundary from analysis /b/: p =.884 /p/: p<.003 No effect for /ba/ Small effect for /pa/. Response=B Looks to P Response=B Looks to P Fixation proportion Category Boundary VOT (ms)