Brain Waves and Button Presses: The Role for Experiments in Theoretical Linguistics

1. Brain Waves and Button Presses: The Role for Experiments in Theoretical Linguistics Alec Marantz Department of Linguistics & Philosophy, MIT KIT/MIT MEG Joint Research Lab

2. Competence static representational accounts for generalizations about structures and relations between sentences and words language knowledge Performance active computational accounts for reaction times and error rates in comprehension and production language use Competence vs. Performance?Standard View

3. Data are DataTheory of linguistic knowledge – the grammar – should be involved in an account of all data • Judgments of grammaticality • Judgments of meaning (e.g., synonymy, entailment) • Reaction times in lexical decision • Reaction times in phoneme monitoring • Amplitude of N400 response to content words

4. Non-issues in competence vs. performance I:Frequency • CAT PORCUPINE • Is frequency “part” of these representations? • (Font size would be proportional to word frequency and time to access/use these representations would be some function of frequency)

5. Non-issues in competence vs. performance II:Categorical distinctions in grammar • Categorical distinctions from grammatical representations do not imply categorical decisions, judgments, or behavior • *[N [Adj [N glori] os] ity] • [N [Adj [N glori] ous] ness] • Categorical grammatical/ungrammatical distinction along one linguistic dimension does not immediately imply categorical judgment from speakers.

6. New Competence/Performance Distinction:Dual Route Theories • Generative grammars instantiate a computational system that maps between sound and meaning • These grammars accurately describe speakers’ knowledge of the connection between sound and meaning • However, suppose speakers have alternative strategies (=computational systems) for connecting sound and meaning for particular “performance” needs (comprehension, production)?

7. Dual Route Theories… • raise the question: • What special data does a linguist have that allow him/her to develop a true account of linguistic computations independent of “strategies” used in language comprehension or production? • In practice: • Linguists must take “strategies” seemingly supported by “psycholinguistic” data as competing theories of linguistic computation • That is: • Data are data and the linguist is responsible for all the data

8. Impact of Experiments on Linguistic Theory • A symbolic importance, a reminder of the potential testability of competing analyses • A constraint on linguistic theory from what might be called the logical problem of language use • Clarification of the concrete mechanisms of language processing in the brain that allows straightforward interpretation of brain and behavioral data

9. KIT/MIT MEG Lab Symbol of potential predictions of linguistic hypotheses

10. I placed a jar in Tennessee, And round it was, upon a hill. It made the slovenly wilderness Surround that hill. The wilderness rose up to it, And sprawled around, no longer wild. The jar was round upon the ground And tall and of a port in air. It took dominion every where. The jar was gray and bare. It did not give of bird or bush, Like nothing else in Tennessee. --Wallace Stevens MEG as symbol: taming the slovenly linguistic wilderness

11. The “logical problem of language use” • Linguistic computations for structures that speakers use must be computable in real time and from the information available to speakers/listeners • Considerations of this “logical problem” support strong locality constraints on information dependencies in linguistic representations, for example

12. The more we know, the more we can discover • Mapping linguistic computation in time and in brain space increases the relevance of psycholinguistic and neurolinguistic experimentation for linguistic theory

13. MEG • Similar to EEG but measures the magnetic field around the electric current source (instead of electric potentials).

14. Magnetoencephalography (MEG) = study of the brain’s magnetic fields http://www.ctf.com/Pages/page33.html

15. Magnetoencephalography (MEG) EEG MEG http://www.ctf.com/Pages/page33.html

16. Outgoing Ingoing Magnetoencephalography (MEG) Distribution of magnetic field at 93 ms (auditory M100) Averaged epoch of activity in all sensors overlapping on each other.

17. 150-200ms M170 200-300ms M250 300-400ms M350 BEHAVIORAL RESPONSE (lexical decision) 400-500ms 200 [fT] 0 200 -100 0 100 200 300 400 500 600 700 [msec] HBM 2003, poster 1345 Visual Word Recognition (Lexical Decision)

18. Activation Competition Selection TURN TURNIP level of activation TURF TURTLE resting level time Stimulus: TURN M350 (i) 1st component sensitive to lexical factors (such as lexical frequency and sound probability) (ii) not affected by form competition, e.g. from phonological neighbors

19. Repetition (i) (ii) Frequency • Stimuli that • speed up early lexical processing • but induce intense competition, delaying RT • elicit faster, not slower M350’s. (Embick, Hackl, Shaeffer, Kelepir, Marantz, Cognitive Brain Research, 2001) (Pylkkänen, Stringfellow, Flagg, Marantz, Biomag2000 Proceedings, 2000) M350 (i) 1st component sensitive to lexical factors (such as lexical frequency) (ii) not affected by competition

20. neighborhood density slows RT (post-M350 compe-tition) sub-lexical probability speeds M350 * * * * n.s. n.s. n.s. n.s. Effect of sound probability/ neighborhood density (n=10) (Pylkkänen, Stringfellow, Marantz, Brain and Language, 2002)

21. Auditory and visual M350 for S1 Auditory Visual RMS (all left hemisphere sensors) M350 field pattern Sagittal view Sagittal view M350 location with respect to auditory M100 A P A P M100 M350

22. Competition effects counteract morphological priming (in cross-modal priming) • gave-GIVE no priming • taught-TEACH yes priming • walked-WALK robust priming

24. MEG Evidence for the morphological complexity of the English Irregular Past Tense Linnaea Stockall, Priya Singh, Pranav Anand, Justin Fitzpatrick and Alec Marantz Dept. of Linguistics and Philosophy, MIT & KIT/MIT MEG Lab

25. Prediction: • gave-GIVE • taught-TEACH • M350 priming, followed by RT inhibition

26. Method Stimuli:4 comparisons • Identity:ghost-ghost vs. trick-ghost • Irregulars • high form overlap:gave-give vs. plum-give • low form overlap:taught-teach vs. warp-teach • Orthographic Overlap: stiff-staffvs. clap-staff 20 test and 20 control items per condition = 320 prime-target pairs Plus 320 fillers (NW-NW, W-NW & NW-W)

27. Method Design: Visual-visual immediate priming (see Pastizzo and Feldman 2002 ) prime + target 450 50 200 0 …2500ms Duration of trial (ms)

28. Results Behavioral Data (n=14) ** * * n.s.

29. Results Behavioral Data: • Significant priming for Identity condition (**p=0.0009) GAVE-GIVE vs. PLUM-GIVE(p=0.03) • Significant inhibition for STIFF-STAFF vs. CLAP-STAFF (p=0.01) • No reliable effect for TAUGHT-TEACH vs. WARP-TEACH (p=0.21) (but trend towards inhibition)

30. Results M350 Priming (advantage of prime condition over control) (n=8) * * * n.s.

31. Results MEG Data: • Significant priming for : Identity condition (*p=0.01) GAVE-GIVE vs. PLUM-GIVE(*p=0.05) TAUGHT-TEACH vs. WARP-TEACH (*p=0.04) • No reliable effect for: STIFF-STAFF vs. CLAP-STAFF (p=0.13) But trend towards priming

32. What do these results say about the kind of stem allomorphy involved in English irregular verbs? • The observed form competition requires that allomorphs compete for recognition – are these: • Stored allomorphs not explicitly related to each other? • Allomorphs derived via special morpholexical rules? • Allomorphs derived via morphologically triggered phonological rules? Brain data provides new empirical texture to these questions. Differential time-course of stem activation and form competition allows one to distinguish levels and types of stored information.

33. HBM 2003, poster 1345 Magnetoencephalographic indices of the effects of morphological family frequency Alec Marantz Department of Linguistics and Philosophy, KIT/MIT MEG Laboratory Massachusetts Institute of Technology Liina Pylkkänen Department of Linguistics/ Center for Neuromagnetism New York University

34. Same number of derivates High frequency derivatives Low frequency derivatives - ist –ize -ism - ic –ize –ism terror magnet Matched for surface frequency Effect of lexical frequency • High frequency words are processed faster than low frequency words. • Prediction of decompositional theories of morphology: cumulative root frequency effects.

35. Should be faster due to high cumulative root frequency Effect of lexical frequency • High frequency words are processed faster than low frequency words. • Prediction of decompositional theories of morphology: cumulative root frequency effects. Same number of derivates High frequency derivatives Low frequency derivatives - ist –ize -ism - ic –ize –ism terror magnet Matched for surface frequency

36. Cumulative root frequency effects for inflection • Response times to a noun depend on the cumulative frequency of the singular and plural (Schreuder & Baayen, JML, 1997) CAT CATS

37. Family size LOW HIGH - ic –ity –ify –head –test –washed - ist acid diary But NO cumulative root frequency effects for derivation Schreuder & Baayen (1997): • Family frequency HIGH LOW Family frequency does not affect lexical decision times. - ist –ize -ism - ic –ize –ism [S&B: Therefore, no decomposition in derivation.] terror magnet High family size speeds up lexical decision times. S&B: this is a late post-lexical effect.

38. Activation Competition Selection TURN TURNIP level of activation TURF TURTLE resting level time Stimulus: TURN M350 (i) 1st component sensitive to lexical factors (such as lexical frequency) (ii) not affected by form competition

39. LOTS OF PHONOLOGICAL NEIGHBORS FEWER PHONOLOGICAL NEIGHBORS HBM 2003, poster 1345 M350: not sensitive to interlexical or allomorphic form competition ** ** SUBLEXICAL FREQUENCY EFFECT COMPETITION EFFECT

40. RT RT slow-down due to competition from highly frequent family members speed-up due to cumulative root frequency HBM 2003, poster 1345 Hypothesis • High morphological family frequency is associated with: = null behavioral effect BUT morphological competition should be distinguishable from phonological competition

41. Same number of derivates High frequency derivatives Low frequency derivatives - ist –ize -ism - ic –ize –ism Matched for surface frequency HBM 2003, poster 1345 Materials: 2 categories of singular nouns terror magnet • Task: Lexical decision

42. 100% 75% 50% 25% HBM 2003, poster 1345 M350 source analysis • Equivalent current dipole analysis • Latencies and amplitudes measured at points where the source amplitude reached 25%, 50%, 75% and 100% of the maximum source strength.

43. n.s. High freq. Low freq. HBM 2003, poster 1345 Results: Lexical decision times (n = 10)No behavioral culmulative root frequency effect(trend toward inhibitory effect of higher frequency)

44. High family frequency HBM 2003, poster 1345 Results: M350 (S1)

45. High family frequency HBM 2003, poster 1345 Results: M350 (S1)

46. High family frequency Extra processing load at the M350 HBM 2003, poster 1345 Results: M350 (S1) Low family frequency • Morphological competition at the M350

47. HBM 2003, poster 1345 Results: M350 amplitude (n=10)

48. HBM 2003, poster 1345 Results: M350 amplitude (n=10) *

49. fine lie - ist –ize -ism loin pine TERROR LINE lane like light lime nine fine lie loin pine LINE lane like light lime nine 1. Difference in the time course of competition High frequency morphological family High density phonological neighborhood (frequency-weighted) • Relationship between target and competitors qualitatively different: difference is due to morphology. DECOMPOSITION • Difference is due to the different phonological and/or semantic properties of the competitors. terrorism TERROR NO DECOMPOSITION DUAL ROUTE THEORY (DECOMPOSITION FOR REGULARINFLECTION) terrorist terrorize

50. fine lie loin pine LINE lane like light lime nine 1. Difference in the time course of competition • Non-decompositional account also predicts interference effects in priming for pairs such as TERRORISM – TERROR. • BUT this is completely unsupported by data – effect is robustly facilitory (e.g. a-d). • Difference is due to the different phonological and/or semantic properties of the competitors. terrorism TERROR NO DECOMPOSITION terrorist terrorize • (a) Marslen-Wilson, W. D., Tyler, L., Waksler, R., & Older, L. (1994). Morphology and meaning in the English mental lexicon. Psychological Review 101, 3-33. • (b) Pylkkänen, L. Stringfellow, A., Gonnerman, L., Marantz, A. 2002. Magnetoencephalographic indices of identity and similarity in lexical access. In preparation. • Gonnerman, L. 1999, Morphology and the lexicon: exploring the semantics-phonology interface, PhD thesis, University of Southern California. • Rastle, K., Davis, M., Marslen-Wilson, W., & Tyler, L.K. (2000). Morphological and semantic effects in visual word recognition: A time course study. Language and Cognitive Processes, 15, 507-538.