Language Perception

1. ABRALIN 22-FEB-05 Language Perception Eva M. Fernández Queens College & Graduate Center CUNY

2. Language Is… PERCEPTION grammar & lexicon SIGNAL MEANING PRODUCTION knowledge about the real world logic

3. Production • I’ll give you my undevoted attention! • You’ll earn her eternal grapefruit. • This restaurant hasn’t been awake very long. • Put the oven on at a very low speed. • We have a lot of churches in our minister. • They roasted a cook. • If you give the nipple an infant… • You ordered up ending. SIGNAL MEANING Structural Assignment Lexical Retrieval Phonological Encoding • phonological fool • a glear plue sky • spattergrain

4. Perception Lexical Access Structure Building Phonological Decoding SIGNAL MEANING Structural Assignment Lexical Retrieval Phonological Encoding

5. Not Present in (Speech) Signal: • phonemes • word boundaries • clause boundaries • location of empty categories • intended attachments for locally or globally ambiguous strings • hidden intents of the speaker! SO HOW COME WE’RE SO GOOD AT DECODING?

6. Visual Illusions when the experiences people report don’t correspond to physical properties of the stimulus very cool… but also very informative about the way the visual / perception system works (which is: modularly)

7. The Hermann Grid Illusion How many grey dots do you see at the “cross-roads”? Source: http://dragon.uml.edu/psych/illusion.html A great page to visit for many more visual illusions.

8. A Face Can’t Be Hollow! A face is always perceived as convex… not concave. http://www.kyb.tuebingen.mpg.de/de/bu/demo/ Max Planck Institut für Biologische Kybernetik

9. Perceptual Illusions also very cool… and also very informative about the way the language perception system works  its modularity ensures its speed and accuracy, which are both in turn compromised when the signal is AMBIGUOUS

10. McGurk Effect by Arnt Maasø, of the University of Oslo: http://www.media.uio.no/personer/arntm/McGurk_english.html

11. Perceptual Displacement andPhonemic Restoration “The state governors met with their respective legislatures convening in the capital city.”

12. “legislatures”, with “cough!” (~ 145 msec) spliced in

13. “legislatures”, intact --- [s] ~ 145 msec

14. Bottom  ~ Top  • study with Broca’s patients (Pollack & Picket, 1964) The apple the boy is eating is red. The girl the boy is chasing is tall. • bait, date, gate study (Garnes & Bond, 1976) Here’s the fishing gear and the ___. Check the time and the ___. Paint the fence and the ___.

15. Structure Building: The Parser • its input is a string of lexical items • its job is to build syntactic structure • its output is sent to a mechanism that decodes meaning • it probably has limited access to information that’s not in the grammar or in the lexicon • it probably operates following a very small set of strategies, grounded on limitations imposed by working memory

16. RSVP Paradigm Center-screen, word-by-word display Timing: N ms per word (here: N = 500 ms) Sentence-recall task

17. The black cat beautiful chased the colorful ball .  The beautiful black cat chased the colorful ball.

18. Black colorful the ball chased cat beautiful the .  Black colorful the ball chased cat beautiful the.

19. The Garden Path Sentence The soldiers marched into the desert surprised the Persian forces. Since Joel always jogs a mile seems like a short distance to him. Carmela put the candy on the table in her mouth. Konstantin understood the problem had no solution. Everybody at the party knew Ann’s date was a total fool. • Local ambiguity • Disambiguation downstream,which goes against parser’s preferences • Reanalysis… or meltdown!

20. The Garden Path Theory Lyn Frazier & Janet Fodor, late 1970s • Minimal Attachment: build the simplest tree • Late Closure: attach locally • Minimal Chain Principle / Active Filler Strategy: posit shortest possible chain / posit gaps for fillers ASAP (the parser is lazy)

21. Minimal Attachment The soldiers marched into the desert surprised the Persian forces. Since Joel always jogs a mile seems like a short distance to him. Carmela put the candy on the table in her mouth. Konstantin understood the problem had no solution. Everybody at the party knew Ann’s date was a total fool. building complex structure = processing cost

22. Late Closure John said Mary will arrive last night. Physicists are thrilled to explain what they are doing to people. Under the glistening tree there was a gift for a boy in a box. Professor Humperdinck artfully avoided looking at the exams of the students that were sitting in his office ungraded. Two sisters reunited after 18 years in check-out counter! attaching non-locally = processing cost

23. Late Closure NP a gift PP in a box Mary saw a gift for a boy… attaching non-locally = processing cost Mary saw a gift for a boy in a box. NP PP P NP for a boy

24. Late Closure John said Mary will arrive last night. Physicists are thrilled to explain what they are doing to people. Under the glistening tree there was a gift for a boy in a box. Professor Humperdinck artfully avoided looking at the exams of the students that were sitting in his office ungraded. Two sisters reunited after 18 years in check-out counter! attaching non-locally = processing cost

25. The RC Attachment Ambiguity N1 N2 The plot concerns the guardian of the prince who was exiled from the country for decades RC La trama es sobre el guardián del príncipe que fue exiliado del país por décadas

26. Cross-Linguistic Differences • N1 attachment rates (%), in studies using questionnaire instruments where: • RC was long • N1/N2 were equal in animacy • Complex NP was in canonical object position for the language

27. Cross-Linguistic Differences As in previous table, for languages other than English & Spanish,listed (for lack of a better strategy!) in alphabetical order:

28. Cross-Linguistic Differences • … could be driven by … • genetic relationship? • syntactic properties? • existence of unambiguous alternatives? • distribution of unambiguous strings in input? • prosody? PROSODY (phonology) PRAGMATICS PARSER

29. Pragmatics? • Grice’s Cooperative Principle “Make your conversational contribution such as is required, at the stage at which it occurs, by the accepted purpose or direction of the talk exchange in which you are engaged” The plot concerns the guardian of the prince who was exiled the prince’s guardian who was exiled * the prince’s guardians who was exiled

30. Long RCs are Informationally Heavy The plot concerns the guardian of the prince … who was exiled. … who was exiled from the country for decades. • Long RC has more lexical content, so it’s more informative. • Does informativeness influence attachment? • RC length effect, confirmed in: • English, Spanish; Arabic, Bulgarian, Croatian, French, German, Hebrew, Hindi, Portuguese… N1 INTERP MORE LIKELY

31. Long RCs are Prosodically Heavy The plot concerns the guardian of the prince who was exiled. The novel’s plot concerns the guardian of the prince who was exiled. The plot concerns the guardian of the prince who was exiled from the country for decades. The novel’s plot concerns the guardian of the prince who was exiled from the country for decades.

32. Elicited Production Fernández, Bradley & Taylor, in prep • N = 8 native US English speakers — 5F, 3M • N = 6  4 sentences,RC Length  Matrix-Subject Weight • RC = 1 versus 3 prosodic words …who was exiled ( from the country for decades ) • MX = 1 versus 2 prosodic wordsThe ( unusual ) plot…

33. 1 The unusual plot concerns the guardian of the prince. The prince was exiled from the country for decades.

34. Acoustic Analysis: Regions Fernández, Bradley & Taylor, in prep Duration: Uniform acoustic signature of phrasal break RC3 Wt S V N1 N2 RC1 The( unusual ) plot concerns theguardianof theprince who wasexiled ( from the countryfor decades )

35. Acoustic Analysis: Regions Fernández, Bradley & Taylor, in prep S ] [ V V ] [ N1 N1 ] [ N2 N2 ] [ RC

36. Fernández, Bradley & Taylor, in prep N2] [RC

37. Region = N2 Fernández, Bradley & Taylor, in prep

38. Elicited Production: Summary Fernández, Bradley & Taylor, in prep MX: F1(1,7) = 2.80, p=.138 RC: F1(1,7) = 11.46, p<.02F2(1,5) = 2.07, p=.209 F2(1,5) = 9.96, p<.05 Interaction MX x RC: F1 < 1, F2 (1,5) = 1.62, p > .25 • … N2 ] [ RC— and nowhere else • Likelihood of break grades with RC length and matrix weight, additively, i.e., with sentence length

39. Questionnaire Procedure Fernández, Bradley & Taylor, in prep • “Reading comprehension test” • 36 targets, 108 fillers (1:3 ratio) • Comprehension question after each sentence • Example of target • The plot concerns the guardian of the prince who was exiled from the country for decades. • Who was exiled? the guardian the prince • Example of filler • The sneaky burglars took all the stereo equipment but overlooked • the computer system. • What was stolen? the stereo the computer

40. Questionnaire Participants Fernández, Bradley & Taylor, in prep • N = 44, Queens College students • US English speakers • Language-history questionnaire, non-nativespeakers excluded/replaced • Rejected/replaced for errors > 15% in fillers

41. Questionnaire Results Fernández, Bradley & Taylor, in prep Relative Clause Length F1(1,40) = 24.95, p<.001 F2(1,32) = 30.12, p<.001 Matrix Subject Weight F1(1,40) = 5.51, p<.05 F2(1,32) = 9.43, p<.01 Interaction F1 < 1 F2 < 1

42. The Implicit Prosody Hypothesis (IPH) “In silent reading, a default prosodic contour is projected onto the stimulus, and it may influence syntactic ambiguity resolution” (Fodor 1998, 2002) the brother of the bridegroom who snores the brother of the bridegroom][who snores

43. Prosody and Syntax Align NP NP N1 PP N1 PP P NP RC P NP N2 N2 RC Selkirk, 1986 the brother of the bridegroom][who often unknowingly snores the brother of the bridegroom who snores prosodic discontinuity el hermano del novio][que a menudo inconscientemente roncaba el hermano del novio ][que roncaba syntactic discontinuity

44. Empirical Support for the IPH • Behavioral evidence on how RCs are interpreted during silent reading • existing dataset: Hemforth et al. (submitted) • Evidence on how the N-of-N-RC construction is produced in discourse-neutral speech • elicited production experiment Do the patterns in the two datasets match up?

45. Behavioral Evidence Hemforth et al. (submitted) • Materials in English and Spanish: • with short and long RCs • N1-N2-RC placed post- and pre-verbally The guest impressed X. Ximpressed the guest. El invitado impresionó a X. X impresionó al invitado. X = the brother of the bridegroom who (often unknowingly) snores el hermano del novio que (a menudo inconscientemente) roncaba

46. Behavioral Evidence Post-VerbalObjects Pre-VerbalSubjects Hemforth et al. (submitted) Who snores? The brother (N1) • Post-Verbal Objects: • Cross-linguistic difference • RC length effect • Pre-Verbal Subjects: • RC length effect reduced • Cross-linguistic difference reduced

47. RC.] N2][RC RC.] N2][RC N2][RC RC][V N2][RC RC][V ENGLISH SPANISH The guest impressed the brother of the bridegroomwho often unknowingly snores. El invitado impresionó al hermano del novioque a menudo inconscientemente roncaba. The brother of the bridegroom who often unknowingly snoresimpressed the guest. El hermano del novio que a menudo inconscientemente roncabaimpresionó al invitado.

48. Experiment: Elicited Production Fernández, Bradley, Igoa & Teira, 2003; Fernández & Bradley, 2004 • Participants, N = 8 per language • English  New York • Spanish  Madrid • Materials, N = 8  4 per language(selected from Hemforth et al.’s 32  4) • Post- and pre-verbal of identical length • RC’s right boundary with same lexical content, whether short or long The guest impressed X. Ximpressed the guest. X = the brother of the bridegroom who (often unknowingly) snores

49. Analyses: N2 & RC’s Verb Fernández, Bradley, Igoa & Teira, 2003; Fernández & Bradley, 2004 • Duration: Presence of Boundary • Pitch movement: Type of Boundary The guest impressed the brother of the bridegroom ][ who … snores.] N2][RCRC.] The brother of the bridegroom ][ who … snores ][ impressed N2][RCRC.][V the guest.

50. Monolinguals: N2 Durations Long RC Short RC 100 ms • Placement × Length InteractionF1(1,14) = 5.77, p < .05, F2(1,14) = 12.37, p < .005 123 ms Post-Verbal 68 ms Pre-Verbal RC-Length  = ENGLISH SPANISH Post-VerbalObjects Pre-VerbalSubjects