The Nature of Mental Representations How is knowledge represented in the Mind?

1. The Nature of Mental RepresentationsHow is knowledge represented in the Mind?

3. Think carefully about the people depicted. What do they look like? Now compare each of their voices How do these representations exist in your mind?

4. The Nature of Mental Representation What does your representation of a penny look like?

5. One of the greatest problems remaining is understanding the nature of mental representation Knowledge Structures: Declarative Memory: facts that can be stated (Knowing that) Procedural knowledge: procedures that can be implemented(Knowing how) How to study mental representation… *Experiments manipulating represented knowledge *Neuropsychology

6. Cognitive MapsTolman 1930’sExperiments by Tolman demonstrated that “representations” exist • Group 1: Rats had to learn maze, reward for getting from start box to end box was food • Made more errors than group 1 • Group 2: no reinforcement for getting to end box • These rats learned the maze without making errors • Group 3: no reward for 10 days. On 11th day food placed in end box for first time. • With one reinforcement ran maze as well as group 1

7. What do Tolman’s Maps look like? Edward C. Tolman (1886-1959) Latent learning goes against standard behavioristic principles, which claim that learning comes only from outcomes

8. Cognitive MapsTolman 1930’s • Conclusions: Learning a cognitive map: • An internal representation of the maze

9. Cognitive MapsBees vonFrisch 1967 • Studied behavior of bees returning to hive after locating nectar • Can form imaginal maps • Can use a symbolic form of communication • Different patterns of dances represent different meanings

10. Cognitive MapsBees vonFrisch 1967 • Round dance: source less than 100 yards from hive • Figure 8 dance: greater distances

11. The Age-old Debate PropositionalversusAnalogical Representation

12. Representational Systems Require: An orderly mapping between the Represented and Representing worlds A mental representation is a system of symbols, conscious or unconscious, that are isomorphic to some aspect of the environment, used to make behavior-generating decisions that anticipate events and relations in that environment C.R. GallistelRutgers University

13. “Words are the images of things” The Greek poet Simonides, c. 500 B.C.

14. “Book” = “Libro” How about Book?

15. Theories of Mental Representation Analogical, Modal, & Embodied Meaning: Meaning is grounded in the sensorimotor processes that result from interactions with the world. Semantic representations retain properties of the sensorimotor states that gave rise to them. Propositional & Amodal Meaning is represented by arbitrary codes that do not retain any of the properties of sensorimotor states that gave rise to them. We don’t store images. Representations resemble abstract form of a preposition. Stored in terms of their deep meanings Relationship between elements/ Element/ Object

16. ? = “Hammer” “xuz1245” Propositional

17. Analogical ? = “Hammer”

18. Propositional Representations for Circle: “A series of points all of which are equidistant from a central point” (Arbitrary code = English, Geometry) “101101101101101010” (Arbitrary code = Binary Digit) “x2 + y2 = r2”(Arbitrary code = Analytical Geometry)

19. Analogical Representation for Circle:

20. Analogical Representations: Thermometers Photographs

21. Cognitive mapsHumans • Landmark knowledge: features: imaginal and propositional • Route road knowledge: specific pathways: procedural and declarative knowledge • Survey knowledge: estimated distances between landmarks: imaginally or propositionally(number)

22. Cognitive mapsHumans

23. Imagery • Mental representation of things not currently being sensed by the sense organs • Often a recollection of something • Can imagine things never experienced • Imagine what it would be like to jump from a plane • Can imagine things that don’t exist • Imagine that Bush wasn’t the current president • Can use any of the sensory modalities • Most research targets visual imagery

24. Kosslyn:We use imagery to solve problems • Which is a darker red: An apple or a cherry? • How many windows in your bedroom? • How do you get to the subway from here?

27. Hypothesis:When people rotate a mental image, the farther they have to rotate it, the longer they will take.Method: Subjects were shown pairs of two-dimensional drawings of three-dimensional objects and were asked whether the drawings in each pair represented the same object in a different orientation or whether they represented different objects.Results: 97% Accuracy.The time it took the subjects to provide the answer was proportional to the angular difference between the two drawings.For example, if the second image was rotated 60 degrees, the subject would take twice as long than if it was just rotated 30 degrees.

31. Experiment • Close eyes and imagine a cube floating in space in front of you • Now: Grasp the Left Front bottom corner of the cube with your left hand and grasp the right back top corner of the cube with your right hand. • Now Rotate the cube so that the corner in your left hand is directly below the corner in your right hand • Describe

33. Using positron emission tomography (PET), an axial brain slice taken 56 mm above the anterior commissure–posterior commissure (AC–PC) line shows activation in the primary motor cortex (M1) when data from the external-action condition were subtracted from those of the internal-action condition. Depending on the strategy used, motor regions of the brain are recruited during mental rotation. The result also shows that the strategy used to accomplish a given task can vary according to previous training, and can be adopted voluntarily.

34. Other evidence suggesting that representations are analogical:1) Mental images of large objects (e.g., an elephant) take longer to scan from left to right than smaller objects2) Details are more difficult to work out in mental images of small things (fruit fly) than in large objects (airplane). For example, Elephant head to tail last longer than Pig head to Pig tail.Similar with map experiments.

35. Distance in image is proportional to time spent scanning

36. Kosslyn 1975 • Imagine : • elephant and a rabbit • Rabbit and a fly • Rabbit and elephant sized fly • Rabbit and a fly sized elephant • TASK: describe details of objects • RESULTS: took longer to describe smaller object of pair • =Perceptual resolution better for larger objects

37. Evidence for Analogical Scanning Stephen Kosslyn (right) and one of his collaborators

38. Perception: longer to scan longer distance than short distances • Experiment to test imaginal scanning • Shown a map of an imaginary island with landmarks • Studied until could reproduce from memory • Critical phase • Hear name of an object (tree) • Picture map, mentally scan directly to object and press a key when have arrived to the object • Hear name of second object, scan to that object’s location • Measured RT

39. Image scaling

40. FindingsDistance & RT are Positively Related

41. Image scaling • Spatial representations can be scanned in the same way that physical percepts can be scanned • Our strategies and responses for imaginal scanning are expected to be functionally equivalent to those we use for perceptual scanning

42. Auditory imaginal representation • Intons-Peterson et al 1992 • Subjects take longer to mentally shift a sound upward in pitch from the low purring of a cat to a high pitched ringing of a phone than to switch from purr to clock tick.

43. Problem:Is it all due to task instructions?Demand Characteristics Zenon Pylyshyn

44. Is the mental image of Kosslyn’s island really map-like? What is farther west, Reno or San Diego?

46. Neuropsychological evidence • Georgopoulos et al 1989 • Single cell recordings in motor cortex of monkeys • Monkey trained to physically move a handle perpendicular to and counterclockwise to a target light • During rotation cortical activity recorded • Later take away handle, cortical activity recorded • RESULTS: activity in motor cortex showed same cortical cells respond when monkeys are anticipating the movements of the rotations

47. L.H.Farah’s patient • Visual imagery(color and shape) distinct from spatial imagery(depth and distance) • LH head injury at age 18 • Lesions in left temporo-occipital regions, right temporal, right inferior temporal • Anomia- could see but could not apply verbal label • Bad at describing color and shape imaginal info • Good with spatial manipulations

48. What processes do we use to represent and manipulate knowledge? • How we represent info influences how easily we can perform cognitive tasks • CMLIX*LVIII

49. What processes do we use to represent and manipulate knowledge? • How we represent info influences how easily we can perform cognitive tasks • CMLIX*LVIII • Easier as 959*58

50. Declarative vs. Procedural • Declarative: knowing that • Procedural: knowing how