Extinction of Conditioned Behavior Chapter 9

1. Extinction of Conditioned BehaviorChapter 9 • Effects of Extinction • Extinction and Original Learning • What is learned during Extinction

2. Effects of Extinction Extinction involves omitting the US or reinforcer • In Pavlovian conditioning, this means repeated presentations of the CS alone • In operant conditioning, this means not giving the reinforcer after the response Two main effects of extinction procedures on behavior • the rate of responding decreases • response variability increases

3. Extinction Increases Variability in Behavior Neuringer, Kornell, & Olufs (2001) Three levers present and Var group had to vary sequence All groups increased variability during extinction

4. Extinction and Original Learning • Spontaneous Recovery • Renewal • Reinstatement • US-devaluation

5. Spontaneous Recovery Rescorla (1996) Rats trained to make 2 different responses for food Lever-press – food Nose-poke – food At end of training phase, rats were making about 17 resp/min After acquisition, the rats were given 2 20-min extinction sessions with each response For one of the responses, the extinction phase occurred shortly before the recovery test (R-No rest) For the other response, the extinction phase occurred 7 days before the recovery test (R-Rest) At end of extinction phase, rats were making about 2 resp/min

6. Spontaneous Recovery Shows importance of passage of time

7. Renewal Bouton & King (1983) Rats were first trained to bar-press for food Then, Context A Conditioning resulted in T suppressing BP response T – Shock During extinction, the rats were divided into 3 groups Group A: CS alone in Context A Group B: CS alone in Context B Group NE: did not receive extinction After extinction, all rats received test trials in Context A

8. Renewal

9. Reinstatement Could be due to context conditioning In initial studies, the excitatory conditioning, extinction and subsequent US exposures were all given in same context Recent research indicates that context is important but not because it permits summation The role of context appears to be to disambiguate the significance of a stimulus that has a mixed history of conditioning and excitation Context has little effect on stimuli that have a history of only conditioning

10. Reinstatement Bouton (1984) Conditioned suppression with rats Phase 1 CS – weak shock CS – weak shock CS – strong shock CS – strong shock Phase 2 No treatment No treatment Extinction Extinction Reinstatement Shock-same Shock-different Shock-same Shock-different Test CS CS CS CS Reinstatement was introduced when the level of responding was similar in all subjects However, some rats got to that point by receiving only conditioning, whereas for others the CS was both conditioned and extinguished

11. Reinstatement

12. Reinstatement For rats that did not receive extinction, it did not matter whether the reinstatement shocks were given in the same context or a different context • shows that contextual conditioning did not summate with the • suppression elicited by the CS For rats that did receive extinction, reinstatement shocks given in the same context produced more suppression than shocks given in a different context • shows that contextual conditioning facilitates the reinstatement • effect According to Bouton, reinstatement US presentations given in the test context serve to restore the excitatory properties of the contextual cues and enable those cues to be more effective in reactivating the memory of excitatory conditioning of the CS

13. US-Devaluation Rescorla (1993) Used devaluation procedure in operant experiment to show that the original R-O association was not abolished by extinction Phase 1 R1 – O1 R2 – O2 R3 – O1 R4 – O2 Phase 2 R1 extinguished R2 extinguished US-devaluation O1 – LiCl Test R1 vs R2 and R3 vs R4 The extinguished responses were re-trained with a third reinforcer before the devaluation procedure to make sure there would be some responding during the final test

14. Effect of Devaluation - Operant Shows that extinction does not eliminate the R-O association

15. What is learned in extinction? The experiments on spontaneous recovery, renewal, reinstatement and US-devaluation all suggest that extinction does not abolish the original learning So why does extinction produce a decrease in responding? Extinction may produce an inhibitory S-R association The non-reinforcement of a response in the presence of a specific stimulus produces an inhibitory S-R association that suppresses that response whenever S is present

16. Inhibitory S-R Association

17. Paradoxical Reward Effects • Overtraining Extinction Effect • Magnitude of Reinforcement Extinction Effect • Partial Reinforcement Extinction Effect

18. Partial Reinforcement Extinction Effect In addition to behavioral effects of extinction, there are also emotional effects (i.e. frustration response) The most important variable that determines the magnitude of both the behavioral and emotional effects of an extinction procedure is the schedule of reinforcement that was in effect before extinction was introduced The dominant schedule characteristic that determines extinction effects is whether the response was reinforced every time it occurred (CRF) or only some of the time it occurred (partial or intermittent reinforcement)

19. Partial Reinforcement Extinction Effect The general finding is that extinction is slower, and involves fewer frustration reactions, if a partial reinforcement schedule, rather than CRF, was in effect before extinction — PREE The PREE was previously thought to be counterintuitive - if a reward strengthens a response, then CRF should produce a stronger response and slower extinction

20. Mechanisms of the Partial Reinforcement Extinction Effect • Discrimination hypothesis • Frustration hypothesis • Sequential memory hypothesis

21. Discrimination Hypothesis • introduction of extinction is easier to detect after CRF than partial reinforcement • with CRF, easier to notice when reinforcement • is absent during extinction • with PRF, not as easy because reinforcement was sometimes absent during training

22. Discrimination Hypothesis While this idea makes intuitive sense, it was discarded a long time ago. Jenkins (1962) 2 groups: Phase 1Phase 2 Group 1: CRF CRF Extinction Group 2: partial CRF Extinction Extinction procedure should be equally discriminable in the 2 groups However, group 2 was slower to extinguish These results suggest that the PREE is not just a discrimination problem but that subjects actually learned something during partial reinforcement that affects the rate of extinction

23. Frustration Theory Early in training, after nonreward, the organism is frustrated and expects not to be rewarded on the next trial. But on some trials following nonreward, the organism is rewarded Thus the stimuli associated with nonreward (frustration) become associated with reward, and the organism learns to respond in their presence – they become SDs for responding. SAppetitive With F = frustration SF: R SF is present during extinction, thus the organism makes many responses.

24. Sequential Theory Early in training, when the organism is rewarded on a trial after one or more nonreward trials, the organism remembers not being rewarded on those previous trials. That memory then becomes associated with reward on the present trial. With N = memory of nonreward SN: R SAppetitive Thus the animal learns to make the response when the memory of nonreward, including sequences of nonreward, is present.

25. During extinction, the stimulus conditions are always those of non-reward. Animals that were on PR schedules should make many responses in extinction. They have learned to respond in the presence of stimuli associated with nonreward (either frustration or memory of nonreward). In contrast, animals that were on CRF, should make few responses during extinction. They have never experienced nonreward, and never learned to respond in its presence. They quickly become frustrated and give up responding.