Chapter 7: The Landscape of Memory: Mental Images, Maps, and Propositions

1. Chapter 7: The Landscape of Memory: Mental Images, Maps, and Propositions

2. Some Questions of Interest • What are some of the major hypotheses regarding how knowledge is represented in the mind? • What are some of the characteristics of mental imagery? How does knowledge representation benefit from both images and propositions? • How may conceptual knowledge and expectancies influence the way we use images?

3. 3 ways the brain creates meaning

4. Mental Representations:Pictures vs. Words • Pictures • concrete and spatial information • analogous to what they represent • Words • abstract and categorical information • symbolic of what they represent

5. Mental Imagery • Internal representation of items that are not currently being sensed • May involve any of the sensory modalities • Imagine a taste, a sight, a touch • Individual differences in creating and manipulating mental images • Use of mental images can help to improve memory

6. Dual-Code TheoryPaivio (1971) • We use two codes to represent information • Analogue (pictorial) codes • Symbolic (verbal) codes • Two codes are linked

7. Visual Codes Processed Differently from Symbolic Codes • Each type of code is affected by different manipulations • Visual interferes with spatial • Verbal interfere with spoken • Sequence matters more for words, not so much for images

8. Evidence for Dual-Code Theory • Brooks (1968) • One group saw a block diagram of a letter • Memorized it • Were asked to mentally travel the letter and indicate if the corner was on the extreme top or bottom Start

9. Evidence for Dual-Code Theory • Brooks (1968) • Second group saw a sentence • Memorized it • Were asked to classify each word as a noun by indicating “yes” or “no” • Verbal task A bird in the hand is not in the bush

10. Evidence for Dual-Code Theory • Brooks (1968) • Participants were then asked to respond in one of two ways • Say “Yes” or “No” • Point to the answer “Yes or No” Why was this important? Yes No No Yes Yes No Yes No Yes No No Yes

11. Evidence for Dual-Code Theory: Interference! For image task, RT was slower when pointing For the symbolic task, RT was slower for the verbal response Different pattern = different processing for different codes

12. Evidence for Dual-Code Theory Participants answered questions about word or picture pairs

13. Longer RT to answer the size question about the word pairs.

14. Propositional Theory • Do not store in form of images • Instead have a “generic” code that is called “propositional” • Store the meaning of the concept • Create a verbal or visual code by transforming the propositional code

15. Propositional Representations

16. Limits of mental images…

17. Test Your Visual Imagery Ability!

18. 1 2 3 4

19. Try Again with Another Design

20. 1 2 3 4

21. Now try it with this figure…

22. Ambiguous figures • People were unable to discover a second interpretation from the image • Then drew the figure and could find the other interpretation • A propositional code may override the imaginalcode

23. Labels matter, too… Hourglass or table Sun or ship’s wheel

24. Results • Participants were asked to draw items seen • Participants distorted the images to fit the labels • This pattern supports the idea that images may be stored propositionally, not as original analog image

25. Mental Imagery Studies • Functional-Equivalency Hypothesis • Mental images are internal representations that operate in a way that is analogous to the functioning of the perception of physical objects

26. Mental Imagery • Shepard & Metzler (1971) • Participants had to decide whether displays had two similar shapes • Some pairs were similar, but rotated to various degrees

27. Shepard & Metzler (1971) ResultsUsing single-cell recordings in the motor cortex, there is physiological evidence that monkeys can do mental rotations

28. Neuroscience and Functional Equivalence • Activation in the frontal and parietal regions occurs when viewing or imagining an image • No overlap in the areas associated with vision • Schizophrenics have difficulty differentiating between internal images and perception of external stimuli

29. AI and Spatial Ability video • What information is essential for navigating in ones environment? • What form might this spatial information take? • What information would a robot need to navigate in an environment and would this differ from what a human would need? • How do animals do it? • rats, bees, and pigeons?

30. Image Scaling • Does the rabbit have whiskers? • Does the rabbit have ears? • Does the rabbit have a beak? • Reaction time to answer is measured

31. Results • It took longer to respond to rabbits paired with elephants than to rabbits paired with flies

32. Image scaling • Asked college students and fourth-graders simple questions about animals • Does a cat have claws? • Does a cat have a head? • Varied the type of instructions • Imagery vs no imagery

33. Results • In imagery condition • questions were answered faster if the attribute was larger • In no imagery condition • questions were answered faster based on distinctiveness of characteristic for the animal, no impact of size

34. Size Judgments • Which is larger, moose or roach? • Which is larger, wolf or lion? • The closer in size, the longer the reaction time

35. Map drawing activity • West acres mall • Red river zoo • Island park • City of Fargo • City of Moorhead • State of North Dakota • State of Minnesota

36. Image Scanning • Kosslyn (1983) • Memorize map • Later asked to scan image • Manipulate distance between items in scan • Hut to grasses • Lake to hut • Measure reaction time

37. Results • support for functional-equivalence hypothesis • Linear relationship between the distance to scan and actual reaction time of participants • Mental images are internal representations that operate in a way that is analogous to the functioning of the perception of physical objects

38. Demand Characteristics • Major criticisms of Kosslyn’s research • Pylyshyn • Only one code, propositional • Results due to task demands • Instructions imply some necessary relationship between the physical distance and time required

39. Demand Characteristics & Mental Scanning • Participants give the experimenters the pattern they expect • Intons-Peterson replicated research but misled experimenters • If experimenter expectations are part of demand characteristics, then leading participants to believe that longer distances would lead to faster responses should alter the results • Evidence supported demand characteristics idea

40. Demand Characteristics & Mental Scanning • Jolicoeur & Kosslyn (1985) • Created a false demand characteristic for a U-shaped function for participants • Proposed that Gestalt principle of proximity makes close points “hard,” and distant points would also take longer • No experimental expectancy effect found • Supported idea that image is being used

41. Johnson-Laird (1983) • Proposed there are three types of mental representations • Propositional representations are pieces of information resembling natural language • Mental imagery are perceptual models from a particular point of view • Mental models are structural analogies of the world

42. Characteristics of a Mental Model • A representation of a described situation rather than a representation of a text itself or the propositions conveyed by a text • The structure corresponds to the functional relations among entities as they would exist in the world • A simulation of events in the world, either real or imaginary

43. Evidence for Mental Models • Kerr (1983) • Studied participants who were blind • Created a tactile Kosslyn map study equivalent • Participants had to study the island, given a physical map to touch • Asked the same scanning questions • Found the same pattern of results—longer distances, longer reaction times

44. Visual Imagery & Spatial Imagery • Visual imagery (images are visual) • Seeing colors • Comparing shapes • Spatial imagery (analog spatial format) • Rotating objects • Aiming and shooting at a target

45. Neuroscience Evidence • Farah (1988) • Brain injury case study (L.H.) • Gave some visual tasks • Color identification, object naming • Gave some imagery tasks • Mental rotation, mental scanning • Poor visual image skill • Normal spatial image skill Thus, both types of imagery must exist

46. In Sum, Researchers Have Proposed • Evidence for analog codes • Evidence for propositional codes • Evidence for mental models • Evidence for mental imagery that is spatial • Evidence for mental imagery that is visual

47. Cognitive Maps • Historically • Tolman – rats • von Frisch – bees • Thorndyke – humans • Gain increased spatial knowledge • Using three types of knowledge • Landmark (special buildings) • Route-road (procedures to get to one place from another) • Survey (global map-like view)

48. Heuristics Affecting Cognitive Maps • Right-angle bias • Streets are drawn at 90-degree angles • Symmetry heuristic • Irregular geographic boundaries are made regular • Rotation heuristic • Tend to “regularize” tilted landmarks to appropriate E-W or N-S axis • Alignment heuristic • represent landmarks and boundaries as better aligned than they really are • Relative-position heuristic • Relative positions of landmarks and boundaries are distorted in ways consistent with conceptual knowledge