Word semantics 2 DAY 27 – Oct 30, 2013

1. Word semantics 2DAY 27 – Oct 30, 2013 Brain & Language LING 4110-4890-5110-7960 NSCI 4110-4891-6110 Harry Howard Tulane University

2. Brain & Language - Harry Howard - Tulane University Course organization • The syllabus, these slides and my recordings are available at http://www.tulane.edu/~howard/LING4110/. • If you want to learn more about EEG and neurolinguistics, you are welcome to participate in my lab. This is also a good way to get started on an honor's thesis. • The grades are posted to Blackboard.

3. Brain & Language - Harry Howard - Tulane University Review

4. Brain & Language - Harry Howard - Tulane University Linguistic model, Fig. 2.1 p. 37 Discourse model Semantics Sentence level Syntax Sentence prosody Word level Morphology Word prosody Segmental phonology perception Segmental phonology production Acoustic phonetics Feature extraction Articulatory phonetics Speech motor control INPUT

5. Brain & Language - Harry Howard - Tulane University

6. Brain & Language - Harry Howard - Tulane University Semantic networks

7. Brain & Language - Harry Howard - Tulane University Lexical semantics 2 Ingram: III. Lexical semantics, §10.

8. Brain & Language - Harry Howard - Tulane University The linkages in such a network are … • associative … • established by the fact that certain words are often used together, such as pig and farm; • these are ‘accidental’, in the sense that there is nothing in the meaning of pig that requires them to be associated with farms; • they are often defined in a free association test, by giving a subject the prime word and asking her to say the first word that comes mind; • or semantic … • the relationships of meaning mentioned yesterday, such as part-whole; • these are necessary, in the sense that a hand is by definition made up of fingers.

9. Brain & Language - Harry Howard - Tulane University Caveat • I grant that the distinction between associative and semantic relationships can be difficult to pin down. • Note that psychologists would call semantic networks ‘semantic memory’, • while linguists would say that most of these networks contain real-world knowledge, which is different from linguistic semantics. • So let us look at an experiment that tries to tease these two domains apart.

10. Brain & Language - Harry Howard - Tulane University Semantic + associative vs. non-associative prime-probe relationsTable 10.4, Moss et al. (1995) Increased priming with respect to control condition in which there is no relationship between prime and probe: unrelated (control) < semantic + non-associative < semantic + associative

11. Brain & Language - Harry Howard - Tulane University Leftovers • The modality of presentation has a large influence. • Auditory priming fades much more quickly than visual priming. • Priming has shown that multiple word meanings are activated before a word is actually recognized. • This reminds of the TRACE model, which is reviewed in the next slide.

12. Brain & Language - Harry Howard - Tulane University An alternative: the TRACE II model

13. Brain & Language - Harry Howard - Tulane University Activation in a semantic network

14. Brain & Language - Harry Howard - Tulane University Some results from brain imaging • I have mentioned a few times a general division of the brain into posterior or sensory cortex (occipital, temporal & parietal lobes) and anterior or motor cortex (frontal lobe). • Should this have any relevance for language? • Nouns vs verbs • Many nouns have ‘high imageability’ and so should require more activation from visual cortex (temporal-occipital lobes) • Verb should require more activation from motor cortex (frontal lobe) • Not all results are consistent, but by and large this is true.

15. Brain & Language - Harry Howard - Tulane University Levels of categorization • On a scale of 1 to 7, rate the following items as a good example of the category furniture. • 1 chair • 1 sofa • 3 couch • 3 table • 5 easy chair • 6 dresser • 6 rocking chair • 8 coffee table • 9 rocker • 10 love seat • 11 chest of drawers • 12 desk • 13 bed

16. Brain & Language - Harry Howard - Tulane University Hierarchy of categories furniture | chair | bench domain-level | basic/prototype | subordinate

17. Brain & Language - Harry Howard - Tulane University Basic is special • Response Times: in which queries involving a prototypical member (e.g. is a robin a bird) elicited faster response times than for non-prototypical members. • Priming: When primed with the higher-level (superordinate) category, subjects were faster in identifying if two words are the same. Thus, after flashing furniture, the equivalence of chair-chair is detected more rapidly than stove-stove. • Exemplars: When asked to name a few exemplars, the more prototypical items came up more frequently.

18. Brain & Language - Harry Howard - Tulane University Basic is really special • 1) It is the highest level at which a single mental image can represent the entire category (you can’t get a mental image of vehicle or furniture). • 2) It is the highest level at which category members have a similarly perceived overall shape. • 3) It is the highest level at which a person uses similar motor actions for interacting with category members. • 4) It is the level at which most of our knowledge is organized.

19. Brain & Language - Harry Howard - Tulane University The fMRI experiment

20. Brain & Language - Harry Howard - Tulane University Why we are interested in vision • The easiest stimuli to use are visual, so we will be gathering information about vision anyway • pictures (name this picture) • text • Reading disorders (dyslexia) have a linguistic component • We are ultimately more interested in audition, however, but perhaps some of what we learn from vision will generalize to it

21. Brain & Language - Harry Howard - Tulane University Early and late visionearly vision is beneath the surface; late vision is on it

22. Constructivist approach ventral ~ identification the stimulation is inherently insufficient, necessitating an “intelligent” perceptual system that relies on inference perception is indirect/multistage process between stimulation and percept memory, stored schemata, and past experience play an important role in perception excels at analyzing the processes and mechanisms underlying perception Ecological approach dorsal ~ visual control of motor behavior the information in the ambient environment suffices and is not equivocal, and thus, no “mental processes” are needed to enable the pick-up of relevant information perception is direct/single-stage process no role for memory or related phenomena excels at the analysis of the stimulation reaching the observer affordances Brain & Language - Harry Howard - Tulane University Norman (2002)

23. Brain & Language - Harry Howard - Tulane University The what / ventral pathway (Palmeri & Gauthier 2004)

24. Brain & Language - Harry Howard - Tulane University Category-specific deficits

25. Brain & Language - Harry Howard - Tulane University The functional organization of the ventral visual pathway and its relationship to object recognition Grill-Spector 2004

26. Brain & Language - Harry Howard - Tulane University Introduction • Humans recognize objects and faces instantly and effortlessly. • What are the underlying neural mechanisms in our brains that allow us to detect and discriminate among objects so efficiently? • Here we examine whether the human ventral stream is organized more around stimulus content or recognition task.

27. Brain & Language - Harry Howard - Tulane University Previous discoveries • Multiple ventral occipitotemporal regions anterior to retinotopic cortex respond preferentially to various objects compared to textures. • Functional imaging studies have revealed that some of these regions respond maximally to specific object categories, such as: • faces (fusiform face area) • places (parahippocampal place area) • body parts • letter strings • tools • animals • These results suggest that areas that elicit a maximal response for a particular category are dedicated to the recognition of that category.

28. Brain & Language - Harry Howard - Tulane University Problems • Comparing activation between a handful of object categories is problematic because it depends on the choice of categories. • While there is maximal activation to one category the activation to other categories is not negligible • Comparing the amplitude of activation to object categories does not exclude the possibility that the underlying representation might not be of whole objects. • Objects from different categories differ in many dimensions and it is possible that the source of higher activation for a category is not restricted to visual differences.

29. Brain & Language - Harry Howard - Tulane University Three ways to represent objects • Kanwisher (2000): ventral temporal cortex contains a limited number of modules specialized for the recognition of special categories (faces, places, body parts) and the remaining cortex is a general-purpose mechanism for the perception of any shape • Haxby et al.(2001): occipitotemporal cortex is organized according to form attributes. The representation of an object is reflected by a distinct pattern of response across ventral cortex, and this distributed activation produces the visual percept • Tarr and Gauthier (2000): occipitotemporal cortex is organized according to the perceptual processes carried out and not by the content of information processed – different cognitive processes require different computations

30. Brain & Language - Harry Howard - Tulane University Experimental tasks • Detection • subjects decide whether or not a gray-scale image contains an object • Identification • subjects discriminate between objects belonging to the same basic level category • for instance, a particular subordinate member of a category (e.g. electric guitar) from other members of that category (e.g. acoustic guitars)

31. Brain & Language - Harry Howard - Tulane University Results • When the category was held constant but subjects performed different recognition tasks (detection vs. identification) similar regions in the human ventral stream were activated. • When the task was kept constant and subjects were required to identify different object categories, different regions of the human ventral stream were activated. • This suggests that the human ventral stream is organized more around visual content than visual process.

32. Brain & Language - Harry Howard - Tulane University NEXT TIME Q7 Continue with word semantics