Recognition

1. Recognition February 26 Troy and Al

2. Outline • Theory of Cognition • Ventral Streams • Experiments

3. Theory of Recognition intensity changes • Bottom-Up Process (Marr 1977) • Top-Down Process (Biederman 1987) objects are made up of component parts (geons) Observed Image Primal Sketch 2 1/2-D sketch 3-D Model depth

4. How to experimentally ascertain recognition? • Thought experiments require self examinationof a subject. In early experiments, the subject was usually a patient with brain damage. In vision, this can cause agnosia: a visual disorder affecting object recognition. Two types: • apperceptive - perception is impaired • associative - association of percept and memory is impaired To determine associative agnosia, the patient must demonstrate: • difficulty recognizing visually presented objects • must demonstrate knowledge of the object through other modalities • must be able to see the object clearly to deccribe its appearance • Today, we can use PET and fMRI on “normal” subjects as they perform some cognitive task

5. An Early ExperimentVision and Firehoses Seeing the world around us is like drinking from a firehose - Kanwisher and Downing (1998) We deal with the firehose, we have attention, which allows selective processing of the information relevant to current goals. • Helmholtz’s (1850) experiment to ascertain visual attention

6. Visual Pathways • Schneider (1969) first postulated anatomical seperation of location vs. recognition ancient retinotectal pathway geniculostriate system • Underleider and Mishkin (1982) location vs. recognition posterior parietal inferior temporal WHERE WHAT • Goodale and Milner (1992) WHAT HOW

7. Visual Pathways

8. Dorsal Pathway Visuo-spatial disorder due to dorsal pathway damage • Patient with parietal injury Newcombe et al (1987) Ettinger (1990) • difficulty choosing correct route through a small 10-choice maze by using a hand held stylus • NO difficulty moving himself through a maze • NO difficulty recalling complex geometrical patterns • NO difficulty carrying out tasks involving short term spatial memory • Some spatial processing affected, object recognition and recollection unimpaired

9. Ventral Pathway Desimone and Ungerleider (1989) and Gross et al (1993) In monkeys, ventral pathway contains cells sensitive to shapes, color, orientation and texture

10. Ventral Pathway Types of Responses: • Selectivity Desimone et al (1984), Tanaka et al (1991) Some TE cells respond only to complex stimuli or stimulus classes • Paradoxical Sensitivity Gross et al (1993) Some TE cells respond to stimuli with few obvious common features • Li, Miller and Desimone (1993) Other cells from TE, perirhinal area 36, and cells going into the frontal lobe that respond to the memory of an object (i.e. objects that are associated in time)

11. Ventral Pathway Damage agnosia • Patient had acute loss of blood pressure causing brain damage (Rubens and Benson 1971) • profound inability to recognize size, shape and orientation of objects • NO difficulty describing objects • NO difficulty recognizing the object when another modality was used (touch or taste) • Patient suffered carbon monoxide poisoning which damaged areas 18 and 19, with 17 intact (Goodale and Milner (1991)) • profound inability to recognize size, shape and orientation of objects (blocks used in experiment) • NO difficulty with hand/finger movements directed at objects (blocks)

12. Site Map Coronal slices from a single subject, arranged posterior (top left) to anterior (bottom right)

13. What’s FFA and PPA? • fusiform face area - ventral occipito-temporal cortex • parahippocampal place area - ventromedial cortical region These areas are studied in tandem since they have opposite response properties

14. So you want to do an fMRI study? Average cost of performing an fMRI experiment in 1998: US$463/hour!!! Average cost of performing a thought experiment: Your Salary CONCLUSION: Unless you are Bill Gates or Michael Jordan, a thought experiment is much more efficient!

15. Be wary of fMRI results! “I wonder whether PET research so far has taken the methods of experimental psychology too seriously. In standard psychology we need to have the subject do some task with an externalizable yes-or-no answer so that we have some reaction times and error rates to analyze – those are our only data. But with neuroimaging you’re looking at the brain directly so you literally don’t need the button press… I wonder whether we can be more clever in figuring out how to get subjects to think certain kinds of thoughts silently, without forcing them to do some arbitrary classification task as well. I suspect that when you have people do some artificial task and look at their brains, the strongest activity you’ll see is in the parts of the brain that are responsible for doing artificial tasks. -- Steve Pinker, interview in the Journal of Cognitive Neuroscience, 1994 Source: Nancy Kanwisher

16. fMRI Setup

17. Lateral Occipital ComplexGrill Spector et al. (2001), using fMRI, conclude: “the ventral pathway contains one region, LOC, which exhibits little selectivity for specific object categories” (but is activated after recognition)

18. Lateral Occipital Complex • Malach et al. (1995) used fMRI to show that LOC has a higher response when the subject views photographs of common objects than when visual textures without obvious shapes are viewed • Kourtz and Kanwisher (2000) used fMRI to show the cue invariance (object format)of LOC response

19. Lateral Occipital Complex • Grill Spector et al (2000) showed neuronal adaptation in LOC: after recognition, higher LOC MR signal: • correlation between subjects’ ability to recognize an object and LOC MR signal. • subjects were trained to recognize images presented for 40ms. fMRI signal was higher after training. • Conclusion: the LOC exhibits little selectivity for specific object categories but is activated after recognition

20. Face Recognition neurons Area of TE immediately adjacent to STP observed by Gross et al. (1972) to have a small population of neurons which respond to “global aspects of the face: the arrangement of the eyes and nose into a facelike configuration”. These cells respond differently depending on • individual monkeys and/or humans • extent of the forehead • presence/width of the eyes • eye direction unit recordings of an STP neuron Bruce et al. (1981)

21. Face Recognition neurons There are experiments that support two ideas: • grandmother cell • circuits of cells

22. Face agnosia Prospagnosia Caused by damage to ventral occipitotemporal and temporal cortex • Component subprocesses in face recognition • different deficits: inability to ascertain age, gender, emotional expressions, and/or identity by viewing a face • NO difficulty demonstrating knowledge of the face through other modalities (auditory, observing clothes being worn) • NO difficulty of perceiving face structure • Pallis (1955) studied a patient with prospagnosia • NO difficulty with recognizing non-face objects • NO difficulty with localization • NO difficulty with memory (short and long term)

23. Visual Attention The effects of attention on the internal representation of a stimulus • Wojciulik et al. (1998) found that activation of the FFA is dependent on attention. That is, the processing of faces can be modulated by selective attention.

24. Visual Attention • O’Craven et al. (1999) used fMRI to analyze object-based attention effects in FFA and PPA: attending to one attribute of an object should enhance processing of the other attributes of that object compared to attributes of irrelevant objects • to distinguish between the effects of object based selection from location/feature based selection, the stimulus looked like

25. Anterior coronal views The task-irrelevant features received more attention when they were associated with the attended object compared to an ignored object ==> object based selection

26. Mentally vs. Visually Applied to Faces (FFA) vs Places (PPA) problem (O’Craven and Kanwisher 2000) Subjects were shown alternating scenes of Face-Place-Face-Place- …

28. Mentally vs. Visually • Faces vs Places (O’Craven and Kanwisher 2000) Results … • Response magnitudes greater for perception than for imagery • Possible to decode the single cognitive from an inspection of fMRI data from individual imagery trials

29. Extrastriate Body AreaDowning et al. 2001 a region of the lateral occipitotemporal cortex EBA in three individual subjects, seen in coronal slices, arranged posterior (left) to anterior (right)

30. Extrastriate Body AreaDowning et al. 2001 Stimulus Space: • Minimizes body vs non-body low level image properties • line drawings • scrambles • Rules out EBA response to any object with rigid subparts connected at flexible joints • Rules out response to all animals

31. Results … OP - object parts FP - face parts SCENE - outdoor scenes WO - whole objects BP - human body parts HAND - hands FACE - faces SO - scrambled objects

32. Sources • Downing PE, Jiang Y, Shuman M, Kanwisher N. “A cortical area selective for visual processing of the human body.” Science. 2001 Sep 28;293(5539):2470-3. • Cohen JD, Tong F. “The Face of Contoversy.” Science. 2001 Sep 28;293(5539):2405-7. • Kanwisher N, Downing P. “Separating the wheat from the chaff.” Science. 1998 Oct 2;282(5386):57-8. • Grill Spector K, Kourtzi Z, Kanwisher N. “The lateral occipital complex and its role in object recognition.” Vision Res. 2001;41(10-11):1409-22. Review • Farah M, Humphreys GW, Rodman HR. “Object and Face Recognition.” The Fundamentals of Neuroscience. Academic Press. 1999. • Goodale MA and Milner AD. “Separate visual pathways for perception and action.”Trends Neurosci 1992 Jan;15(1):20-5 • Kanwisher N. Kanwisher Lab Homepage: http://web.mit.edu/bcs/nklab/ • Downing P, Liu J, Kanwisher N. “Testing cognitive models of visual attention with fMRI and MEG.” Neuropsychologia. 2001;39(12):1329-42. Review. • Culham JC and Kanwisher NG. “Neuroimaging of cognitive functions in human parietal cortex.” Curr Opin Neurobiol. 2001 Apr;11(2):157-63. Review. • O’Craven KM and Kanwisher N. “Mental imagery of faces and places activates corresponding stiimulus-specific brain regions.” J Cogn Neurosci. 2000 Nov;12(6):1013-23. • Haxby JV, Gobbini MI, Furey ML, Ishai A, Schouten JL, Pietrini P. “Distributed and overlapping representations of faces and objects in ventral temporal cortex.” Science. 2001 Sep 28;293(5539):2405-7.