Representation – Dual Codes

1. Representation – Dual Codes

2. Re: Catching & holding attention If you were going to offer a series of informational sessions to miners regarding health . . .

3. Lessons for 'cranky' miners Australian coal miners are being given lessons in foreplay and the menopause in an attempt to boost productivity. Managers at the Bulga pit, north of Sydney, say the so-called toolbox talks help workers to understand their wives, making them happier and healthier. Mining firm Xstrata says the scheme has been so successful that it wants to extend it to other mines. The briefings are on a different topic each month and have included advice on fatigue, nutrition and heart disease. Company bosses say giving their predominantly male employees lessons on the menopause and foreplay gives them a healthy sex life, which in turn makes them happy, productive workers. "We have to look at the lifestyles of our employees, making sure they are fit and healthy at work, but also fit, healthy and happy at home," Xstrata spokesman James Rickards told Reuters. Course co-coordinator Tammy Farrell, from Core Health Consulting, told the BBC that she was trying to promote communication and make the miners more aware of their health. Earlier, she told the Sydney Morning Herald that the miners had been "extremely attentive" when she advised them to "start exploring their wives like they did when they were 18". "They snapped up all the flyers after the talk so we've obviously got some cranky men with cranky wives out there who want some help," she told the paper. Story from BBC NEWS: http://news.bbc.co.uk/go/pr/fr/-/2/hi/asia-pacific/7061475.stm Published: 2007/10/25 14:08:30 GMT

4. Psy 348: Knowledge representation: verbal versus pictorial Two types of knowledge Declarative = “Knowing that . . .” [Gilbert Ryle] Examples Procedural = “Knowing how to . . .” Examples These different types may generate different kinds of mental representations Since declarative knowledge tends to be “facts,” it might well be represented verbally or symbolically Procedural knowledge often involves manipulation of objects & thus might often be represented via images

5. Two major theoretical positions regarding how knowledge is represented mentally – Dual Code and Propositions

6. Dual Code -- Paivio (1971) • We use two codes to represent information • The two codes are linked, so we can coordinate them &/or switch between them • Analogue code – (often called pictorial) similar to perceptual repre’s • Verbal code - arbitrary symbols (e.g., letters, words, #’s) represent items

7. Representations of a Carpet Analog/pictorial representations Symbolic/verbal representation Carpet, Rug

8. (1) Verbal code Historical context: Nonverbal thought [oddly, often called “imageless” in English] (Kulpe at Wurzburg) versus linguistically based symbols (Wundt at Leipzig). Kulpe had been one of Wundt’s assistants, but when he start his own lab at Wurzburg, he abandoned one of Wundt’s fundamental tenants: that consciousness & complex thoughts cannot be studied directly – we must focus solely on momentary sensations & feelings, represented & expressed via linguistic symbols. Wundt “won” that early debate, in the sense that most early psychologists accepted his tenants, and these evolved into a strong pro-linguistic bias in western psychology that also rejected visual or other sensory images until the 1970’s. Abstract concepts are especially strong here: love, justice, democracy . . . Verbal info usually represented symbolically & arbitrarily just like most words & numerals Examples . . . !

9. (2) Analogue / Pictorial code • Concrete objects,mental maps and geometric shapes are notable here . . . • This representation is often called “analogue” because it is analogous to • the real-world object that it represents • It also behaves in ways that are similar to perceptions • functional-equivalence hypothesis • Mental images are internal representations that operate in a way that is analogous to the functioning of the perception of physical objects • -- First proposed by Shepard and Kosslyn

10. Historical context: Edward Tolman (1948 ff) – Mental Maps How do rats successfully navigate their surrounding – for example, a maze containing a hidden reward? Tolman suspected that rats would build mental maps of the maze as they investigated it (forming a mental picture of the layout of the maze). During the heyday of Behaviorism in the U.S., many of his colleagues thought that rats would learn to navigate the maze through stimulus-response, associating particular cues with particular outcomes (e.g., taking this tunnel means I get a piece of cheese) without forming any mental picture of the maze.

11. To test these ideas, Tolman and his colleagues trained rats in a maze which offered them many different tunnels to enter first. One of the tunnels twisted and turned but consistently led to the reward, and the rats quickly learned to go down that tunnel. Then the experimenters blocked the entrance to the reward tunnel. What would the rats do?

12. Tolman found that most of the rats picked a tunnel that led in the direction of the food, instead of one close to the original reward tunnel. The evidence supported the idea that rats navigate using something like a mental map.

13. Tolman sources: http://undsci.berkeley.edu/article/0_0_0/whatisscience_10 Illustrations & notes for the foregoing slides http://psychclassics.yorku.ca/Tolman/Maps/maps.htm The rat experiments described in great detail Incidentally, Christopher Green’s site, “Classics in the History of Psychology” is an exceptionally useful site for looking up info about the “great ideas” and/or the “great people” in psychology: http://psychclassics.yorku.ca/

14. Although Tolman’s ideas & research findings were provocative, they nevertheless languished in the ocean of Behaviorism that dominated American psychology until the late 1970’s. His contemporaries were convinced that his findings were simply some kind of anomaly that surely could be explained by Stimulus-Response theory. That is, that there were no “mental maps,” neither in animals nor in humans, and that the rats in his studies must have been learning the layouts of the mazes through some kind of S-R effects that Tolman wasn’t noticing. An important reminder to all of us: once we have a strong idea in which we deeply believe, we simply ignore evidence to the contrary, until the evidence becomes completely overwhelming . . .

15. That overwhelming evidence began to appear starting in 1971 in a sensational series of research papers published by Roger Shepard and his students that supported the idea that humans can and do form and use mental images. At first, of course, these findings too were rejected, both by the behaviorists, and by people who were convinced that mental representations, if they exist, could only be verbal/symbolic codes.

16. Mental rotation – Shepard & Metzler – Shepard & Cooper Form a mental image of an object, then imagine that is it rotated to a diff position; “rotating” the imagined object takes time, just like rotating a real object. Hence, based on Shepard & Metzler’s work & on Kosslyn’s work, along with many others: the “functional equivalence” hypothesis – Mental representations are functionally equivalent to real objects and mental imagery is strongly analogous to perception. Many imagery theorists insist that the processes are essentially the same. .

17. Mental rotation of “2-D” objectsShepard & Cooper

18. Mental Rotation of "3-D“ objects • Participants were presented with pictures of 3-dimensional objects • Task: determine if two objects are identical except for orientation • Participants have the impression of mentally rotating the objects [Shepard & Metzler, 1971] 10.1.2

20. Shepard & Metzler (1971) Results

21. Mental Rotation Studies Demonstrate -- Mental comparisons of pictorial objects is an active process i.e., “rotation” of the mental objects -- Response times are proportional to degree of rotation -- Speed in 2D = Speed in 3D -- Images are treated as though they are “Mental Sculptures” i.e., as though they are real, 3-D objects: we “rotate” them to see if they match, just like we might do with real objects

22. Mental imagery – Kosslyn (vision –image scaling & scanning); Also Intons-Peterson; Farah; Finke, et al --representing things not currently being sensed (i.e., “in memory”) --can represent information in any sensory modality Examples: Kosslyn’s image scaling & scanning Scanning the images takes time, just like scanning real-world objects

23. Image scanning – Kosslyn

24. Image Scaling Kosslyn (1975) • Examine how participants scan and use images • Some participants imagine an elephant next to a rabbit • Others imagine a rabbit next to a fly • Then answer questions about the rabbit • Does the rabbit have whiskers? • Does the rabbit have ears? • Does the rabbit have a beak? • Reaction time to answer is measured • If RT’s are the same in both conditions, then verbal code • If RT varies depending on the condition, then some sort of pictorial image/code + scaling must be involved

25. Image Scaling Judgments faster for rabbits next to smaller creatures (larger visual image) Than for rabbits next to larger creatures (smaller visual image)

26. Image Scaling (Moyer, 1973) • Which is larger, moose or roach? • Which is larger, wolf or lion? • When objects are different sizes, RT is faster than when they are similar sizes. • Similar results when making comparisons of actual physical objects while looking at them • If we used a verbal code to represent objects, then RT to make the comparisons would be about the same, since no imaginary “looking at” the objects would be involved.

27. Image Scanning Kosslyn (1983) • Memorized map • Later asked to scan image • Manipulate distance between items in scan • Hut to grasses • Lake to Hut • Measure reaction time

28. Image Scanning • Linear relationship between the distance to scan and actual reaction time of participants • Support for functional-equivalence hypothesis • Mental images are internal representations that operate in a way that is analogous to the functioning of the perception of physical objects

29. Summary: Two codes, one symbolic/verbal and one analogical/pictorial A compelling theoretical position, with lots of empirical support Nevertheless not all of the characteristics of the referent are retained. Furthermore, mental images are not particularly compelling, and lack the complexity, vividness, and dynamism of real-world perception. And finally, some people object to the “demand characteristic” that seems to inhere in mental imagery research: Especially in Kosslyn’s studies, participants were usually told to “scan” the image . . . However, research designed specifically to test for demand characteristic effects mostly have not found them, so that objection has largely been nullified.