Extinction

1. Extinction How do animals “know” when a schedule is on extinction?

2. Remember • Operant conditioning extinction differs from classical conditioning extinction • Responds decreases to near zero for both • Operant conditioning: • Transient increase • Extinction induced aggression

3. Partial Reinforcement Extinction Effect: PREE • Extinction occurs at different rates depending on the schedule: • Continuous reinforcement: FAST extinction • Partial reinforcement schedules: SLOWER extinction • Variable schedules show slower extinction than fixed (rate or time) schedules. • PREE used to describe greater persistence in instrumental responding during extinction after partial (or intermittent) reinforcement training • Faster extinction after continuous reinforcement training. • Partial reinforcement schedules show RESISTANCE TO EXTINCTION

4. Other Extinction Effects • magnitude reinforcement extinction effect • Less persistence of instrumental behavior in extinction following training with a large reinforcer • More persistance of responding with a small or moderate reinforcer. • Effect is most prominent with continuous reinforcement. • overtraining extinction effect • Less persistence of instrumental behavior in extinction following extensive training with reinforcement (overtraining) • Faster extinction following moderate levels of reinforcement training. • Again, effect most prominent with continuous reinforcement

5. Other Extinction Effects • reinstatement • Recovery of responding to an extinguished stimulus • produced by exposures to unconditioned stimulus or reinforcer • renewal • Recovery of excitatory responding to an extinguished stimulus • produced by shift away from the contextual cues that were present during extinction.

6. Why do we get these effects? • Behavioral momentumand Sequential theory • Trouble with discrimination • Frustration

7. Behavioral Momentum • Suggests that PREE occurs because the animal has a high momentum of responding and it is more difficult to stop this momentum • Timberlake and Lucas 1985: • Ball bearing studies • Rolled ball bearing across cage; rats had to let it go past to receive reinforcer • Played with the ball bearing, slowing reinforcement • During extinction (ball bearing but no food): played with ball bearings MORE • Does suggest that animals show strong patterns of behavior that may interfere and thus slow the extinction process • But not a complete explanation

8. Discrimination and Frustration • Discrimination hypothesis: • Mowrer and Jones 1945 • In order for subjects’ behavior to change during extinction, the subject must be able to discriminate the change in reinforcement contingencies • With CRF: This is immediately noticeable • With PRF: not immediately noticeable • More discriminative on fixed schedules • Less discriminative on variable schedules • Evidence does not completely support this

9. Generalization Decrement Hypothesis • Capaldi, 1966 • Generalization decrement: decreased responding observed in generalization test when test stimuli become less and less similar to training stimulus • Responding during extinction is weak if the stimuli present during extinction are different from those during the reinforcement phase • Responding during extinction is STRONG if the stimuli present during extinction are very similar to those during reinforcement phase.

10. Generalization Decrement Hypothesis • Large generalization decrement when schedule moves from CRF to EXT • Subject never experienced situation in which some of its responses are not reinforced • Not been taught to keep responding in absence of a reinforcer • Small generalization decrement when schedule moves from PRF to EXT • Subject has experience in situation where some of its responses are not reinforced • HAS been taught to keep responding in absence of a reinforcer

11. Sequential Theory • sequential theory: memory of reward vs. non reward • Cognitive theory • Fast extinction after CRF • Extinction occurs quickly because the instrumental response has NOT been conditioned to the memory of nonreward • Slow extinction during PREE • extinction is slowed after partial reinforcement because the instrumental response becomes conditioned to the memory of nonreward.

12. Bottom Line • Rate of extinction is affected by: • Discriminability of the situation • Previous experience with non-reinforced responding • Possibly, memory of non-reinforced responding • Why frustration? • Not getting what expected • Because of contingency between response and reinforcer: • Subject “controls” responding, and thus reward • During EXT: this contingency is disrupted • Subject becomes “frustrated” in that • Tries other responses • Increases magnitude of responding • Increases intensity of responding • Pushed into fear/flight/fight sequence: aggression is elicited

13. Avoidance:

14. Avoidance Tests • Negative reinforcement = removing a stimulus to INCREASE a behavior • Negative reinforcement = • escape: a response removes something • avoidance: a response prevents some event • Procedure for studying negative reinforcement and avoidance: Discriminated avoidance: • a response CANCELS a shock • Organism is responding for food reinforcers • When light comes on, must press another lever to AVOID the shock • if the response does not occur during the S+ the stimulus is followed by a shock • if the response does occur during the S+, the shock is cancelled • thus: signal or sD for shock • if this were an escape: response could also occur DURING the shock to shut off shock

15. Two Avoidance Procedures: • Sidman Avoidance: • the response POSTPONES or DELAYS the shock • thus: only temporary solution • must keep responding to keep delaying the shock • results in lots of responding • again: some signal may be used to signal when must respond • Herrnstein and Hineline Procedure: • the response reduces the rate of the shock • note: note delay or cancel, just slows down rate of delivery • the response switches the schedule of shock to a lower rate • Note: cannot entirely AVOID shock in this procedure: • once animal receives shock on lowered schedule, reverts back to original schedule • animal must respond again to switch schedule again

16. Characteristics of Avoidance Behavior • extremely variable: • from subject to subject • from session to session with SAME subject • procedure to procedure • choice of response is important • determines how quickly will learn contingency • how well learning is maintained • 1-way vs 2-way shuttle avoidance tests: • 1-way shuttle: run to other end w/sD • 2-way shuttle: run to opposite end w/sD • rat will learn to run to other end of shuttle box when sD comes on to avoid shock • difficult time learning 2-way shuttle avoidance

17. Characteristics of Avoidance Behavior • Species specific Defense reactions: • behaviors which animal does naturally in time of danger • includes: freezing, fleeing, fighting • Why? • animal has innate behaviors does when avoiding noxious stimulus- • can't make it go against its nature

18. Characteristics of Avoidance Behavior • Negatively reinforced behavior is difficult to extinguish: • escape behaviors take long time to go away • e.g.: rat in 1-way shuttle still runs when light comes on-even after hundreds of EXT trials • BUT: will extinguish quickly if animal can detect change from conditioning to EXT situation

19. Negative Reinforcement in Humans • most often "reinforcement" technique used in real world • often used because is cheaper, easier, more natural • produces "bad" side effects: avoidance responses to sD = boss, principal, spouse, etc.

20. Theories of Avoidance: Two Factor theory • Two things happen during avoidance conditioning: • animal learns to fear S+ via class. cond'ing • CS (light)---> US (shock): UR (fear) • animal learns to fear light via pairing with shock • animal will then learn a response to AVOID shock and thus remove/lessen their fear • thus: not getting shocked reduces fear that was signaled by the CS • experimental evidence: • on initial training trials: • light/CS produces physiological symptoms of fear • escape response results in decrease in these physiological symptoms • on later trials: • little or no evidence of physiological fear with CS presentation • suggests fear has been reduced/replaced by the escape response • in sense: forms a negative feedback loop

21. Two Factor theory in Humans • many ineffective and/or irrational fears • often involve avoidance responses due to original fear • maintained by decrease in fear • e.g., banging two sticks to keep the tigers away • Symptoms of obsessive/compulsive disorders: • compulsions = repeated, stereotyped, ritualized actions • individual feels compelled to engage in them • obsessions = compulsive thoughts (no actual actions) • many, many examples of this • can begin to interfere in life

22. Problems with 2-factor theory: • Signs of fear dissipate w/time: • as animal gets "better" at avoidance response • thus: no fear to be avoided • the CS is not as important in avoidance learning as 2-factor theory states: • animals can learn to avoid in a discriminated avoidance situation long before there is any sign that they are responding to/detecting the CS

23. Herrnstein and Hineline: Test of 2-factor theory • Herrnstein and Hineline procedure: • not use any CS, but the animal still learns to lessen/avoid the shock • Test of the theory: • two groups of rats used • Group 1: can turn off light, but still shock • Group 2: can turn off shock, light still on • 2-factor theory would predict that Group 1 should respond more, because this would be cancelling the CS that produces fear • results: group 2 responds much more accurately, faster

24. Alternative: One-Factor Theory • responses occur whenever they reduce the rate at which aversive events occur • when a CS is present: only providing information about the effectiveness of a response • fear may be a by-product of avoidance training, but not crucial to learning/ maintaining an avoidance response

25. Evidence for One-Factor theory • Almost postulating a "cognitive" theory of avoidance: • Seligman and Jonston (1973) did postulate cognitive theory: • like Rescorla Wagner theory in that deals with predictability • Basic premise: • learning occurs only when discrepancy between observation and expectation • subjects' behavior will change in avoidance task whenever there is a discrepancy between expectancy and observation

26. Evidence for One-Factor theory • Two important expectations in avoidance task: • expectation about consequences of a response • .expectation about consequences of not responding • Data support One-factor theory • on trial 1: no expectations • On trial 2 (and more): expectation about what will happen • no shock will occur if response is made • shock will occur if no response is made • animal prefers no shock to shock- so responds • Contingency is what is important in avoidance, fear is by-product!

27. Evidence, con’t. • as long as animal continues to respond- no shock • not know when extinction occurs- no sampling • only stop when learn situation has changed • Thus: to EXT responding: • Must use response blocking or flooding: • present sD, but prevent R from occurring • thus animal learns that shock no longer comes • animal stops responding in presence of sD

28. Flooding • To extinguish an inappropriate response: must make contact with "changed reinforcement or punishment" situation • sometimes used as alternative to systematic desensitization • flood with presentation of fear-provoking stimulus • Again, no actual consequence occurs • Continue presentations until the response is extinguished • Problem: may "scare the patient to death"