Paradoxical Effects of Reward

1. Paradoxical Effects of Reward • Overtraining extinction effect: more training leads to faster extinction • Reinforcement magnitude effect: Big rewards lead to faster extinction • And, of course, the partial reinforcement extinction effect (PREE)

2. Paradoxical effects of reward: Why? • Discrimination hypothesis: Nonreinforcement is easier to detect after CRF than PRF.

3. Discrimination Hypothesis: Test CRF  CRF  EXT vs. PRF  CRF  EXT

4. Paradoxical effects of reward: Why? • Discrimination hypothesis: Nonreinforcement is easier to detect after CRF than PRF. • Frustration hypothesis (Amsel): animals learn to make response as a reaction to nonreward. • Sequential theory (Capaldi): The memory of nonreinforcement becomes a cue that elicits responding.

5. Stimulus Control

6. Stimulus Control of Behavior • Having stimulus control means that the probability of the behavior varies depending upon the stimuli present • Most of our behavior is under stimulus control • A person that contributes to charity generously while in church may watch every penny spent while at work

8. Discrimination and Stimulus Control • Discrimination is demonstrated when differential responding occurs to two or more stimuli. Reynolds (1961) Train Test

9. Generalization • Generalization is when responses to one stimulus occur to other, usually similar, stimuli • Generally, as the training and test stimuli become more different responding will decline, producing what is called a generalization gradient

10. Generalization GradientGuttman & Kalish (1956) • pigeons reinforced for pecking a 580 nm lit key (orange-yellow) (S+) on a VI schedule • A test session was then given where many different colored key lights were presented in extinction S+

11. Interpreting Generalization Gradients Pigeons trained to peck a moderately bright light (S+) to get food. (S- = dim light) After asymptote is reached, present occasional non-reinforced probe trials at various wavelengths or levels of brightness.

12. Excitatory andinhibitory gradients Pigeons trained to peck at a 800 hz tone (S+), with a 500 nm light S-.

13. 1000 Hz Tone S+ / 950 Hz Tone S- 1000 Hz Tone S+ / No Tone S- 1000 Hz Tone always on

14. Peak Shift Effect – Hanson (1959) Control group: 550 nm Light S+Experimental group: 550 nm Light S+ / 590 nm Light S-

15. S+ S- Spence’s Theory to Account for Peak Shift

16. Interdimensional discrimination Discrimination: S+ = 555nm Light; S- = Tone

17. How do we learn discriminations with complex stimuli?

18. A+B A B How do we learn discriminations with complex stimuli?

19. Complex Discrimination: Example Pre-exposure Devalue Test -- Saline-LemonLiCL Sucrose-Lemon? Lemon Saline-LemonLiCL Sucrose-Lemon? Another example… Pre-exposure Devalue Test -- Saline-LemonLiCL Sucrose-Lemon? Sal-L/Suc-L Saline-LemonLiCL Sucrose-Lemon?

20. Complex Discriminations: Mechanism # 2 the method of pre-exposure matters… Pre-Exposure: AXBXAXBX | CXCXCXCX Devalue: AXLiCL Test: BX? CX? Question: How much does aversion generalize to BX and CX? A = lemon B = salt C = sucrose X = quinine Mondragon & Hall (2002)

21. What’s going on? • Juxtaposition of stimuli clearly matters • But why? AXAXAX… produces habituation to AX Remember: expected things are less salient or associable AXBXAXBX…. A B

22. Treating Different Stimuli Alike: Categorization “Categorization can be viewed as the ability to treat similar, but not identical, things as somehow equivalent, by sorting them into their proper categories and by reacting to them in the same manner” (Huber, 2001) • Classical view: categories united by a defining feature or features • But Consider: Oak leaves v. Non-oak leaves Chairs v. non chairs

23. What is “Chairness” “family resemblance”

24. "A pigeon pecks rapidly at a small photograph of Harvard Yard containing trees, buildings, people, sky. After a few seconds, a hopper of grain appears and the pigeon eats. Now the scene changes to a treeless Manhattan street. The bird emits a few desultory pecks, then turns away and paces about. After a minute or so, a picture of a leafy suburban garden appears and the bird begins pecking again." (Shettleworth 1998) Categorization Experiments Train Test Scenes with Trees + New Set tree scenese Scenes w/o trees - New Set of no-tree scenes

25. Other categories pigeons can form • Aerial v. non-aerial photos • Chairs • Humans • Cars • Defective pharmaceutical capsules! • Oak leaves versus other leaves

26. Human v. Non-Human

27. How do they do it? • Exemplar theory: remember category members and then generalize. • Vaughn & Greene 1984: pigeons can remember no less than 320 individual slides! Outdoor scenes randomly assigned to + or –

28. Exemplar theory: more evidence • Cook (1990) • Birds versus Mammals used in slides • Real Category Group: Birds v. Mammals • Pseudocategory Group: Random Bird & Mammals versus Random Birds & Mammals

29. Feature Theory • Individual features acquire associative value. • Response rate to stimulus depends on total expectancy (V) evoked.

30. Feature Theory: Evidence Cerella (1980): Train: Charlie Brown +, other characters – Test: Keep all features intact, but alter whole

31. Prototype theory • Abstract the “ideal” (or average) category exemplar. • To test: train with only extreme exemplars, test with average of extremes.

32. Prototype Theory in Humans Posner & Keele 1968

33. Conclusions: • Not clear whether birds can extract abstract concepts in categorization experiments • Birds may use features and exemplars • Another animals may be capable of more complex feats.