One Behavioral Approach to College Teaching:

1. One Behavioral Approach to College Teaching: A Progression from a Lecture/Text Course to Programmed Self-instruction MABA 2006, October 20–21 Carbondale, Illinois Jack Michael Western Michigan University This PowerPoint presentation has been modified from the way it was presented as an ABA tutorial in 2005. I have put many text items on a single slide because you can display the slides for as long as you want, and can go back to previous slides. This presentation will be on my web site, but I will also put the presentation on the web site as it occurred at the tutorial.

2. The instructional methodology advocated here works best for what I call content courses, courses with fairly clear instructional goals related to a specific content or subject matter. Students are expected to acquire a speaking/writing/listening verbal repertoire in the subject matter. Such courses are often aimed primarily at preparing the student to take more advanced courses in the same area. Text material covering most of the content is usually available. Typical instructional methodology is to assign text material, and give lectures over the text and related material. Repertoire acquisition is usually assessed with in-class exams typically given more often than twice a semester, but seldom as often as once a week. Multiple-choice if large class, maybe some short answer questions. Students learn by studying the text outside of class, listening to lectures, and studying lecture notes.

3. Also in the humanities, education, and business. Third and fourth year courses in these areas may also be primarily content courses. Many first and second year courses in the natural, behavioral, and social sciences are of this type. I am not recommending this methodology for courses primarily concerned with skills in athletics, music, art, or dancing. Nor for courses concerned with scientific or engineering laboratory skills. Nor for those primarily concerned with interpersonal skills; or self-awareness, personality development, and so on.

4. Eleven years as a college student at UCLA: chem major 43-44; psych major 46-49; experimental psych grad student 49-55. Background Factors: As a College Student • As an undergrad and as a grad student I had a very positive attitude toward college: I valued what I was learning in most courses and I admired most of the teachers. • Most courses were good examples of the text-lecture method and the teachers were good models of effective college profs. • No matter how the course was taught, I felt that I was learning important things that were new to me. For my behavior as a student: The nature, frequency and total time spent studying was mainly determined by the nature, frequency, and extensiveness of the exams. And this relation depended upon the importance of exams for the course grade.

5. My first five years as a college teacher: 55-57 at Kansas University; 57-60 at University of Houston. As a College Teacher During this time I taught several undergrad courses (30 or so students) and also grad courses (10 or so students) I think I taught like I was taught at UCLA. I used the lecture to clarify difficult text material, elaborate, provide additional examples, and introduce some new material (if possible a behavioral interpretation from SHB). I encountered students' poor note taking skills as a problem with lecture as a source of information. And even students’ reactions to text were often unsatisfactory; they did not react to the material in the same way that I did. Exams as the main determiner of study was confirmed, and also the course grade as the main motivation determining behavior related to the course.

6. Common academic beliefs which I later considered quite counterproductive. 1. Very poor academic performance was largely due to students being intellectually incapable and/or lazy. Very good performance was due to students being very bright. 2. Grades and exams should not be emphasized. Students should be working because of an intrinsic interest in the subject matter (which would be enhanced by good lectures) and for long term goals (although everyone knew, but regretted, that studying was related to exams and the course grade. 3. Good teaching is mainly good classroom performance--good lecturing (and later good visual displays--overheads, etc.). 4. From SHB (and later from some other behavioral teachers) I thought that learning should involve as little aversive control as possible. Learning should be fun and easy. Not quite right. I have often called items 2, 3, & 4 the romantic approach to college teaching. I think I got this term from Scott Wood.

7. Two important developments in behavioral approaches to college teaching. F. S. Keller's Personalized System of Instruction (PSI): Developed in Brazil in 1963. • Programmed Instruction: B. F. Skinner (1954) The science of learning and the art of teaching. Harvard Educational Review, 24, 86-97. (Described in next slide.) I first learned about PSI in Keller's presentation at the 1963 APA conference. It had no name at that time. In 1965 Keller began using it in an undergraduate course at Arizona State U.(where I was teaching). The first widely available written version was in the first issue of JABA: Keller, F. S. (1968) "Goodbye Teacher. . .". Journal of Applied Behavior Analysis,1, 79-89. Two other behavioral developments in education, aimed at k-12 education, but not irrelevant to higher ed.: Siegfried Engelmann's Direct Instruction, and Ogden Lindsley's Precision Teaching.

8. Programmed Instruction • PI was a major behavioral application to education--breaking knowledge into small components and building a repertoire, prompting to keep error probability low, then fading the prompts. Frequent reinforcement of small steps. Example: The Holland & Skinner program The Analysis of Behavior, available as a programmed textbook. But not easily incorporated into college teaching. Still had to give exams and base grades on the exams. No teaching machines until personal computers in the early 80s. Almost no programs available for typical college courses, and a teacher could not easily produce one for a specific course. Nevertheless, PI was important for conceptualizing knowledge. Very important side effect: Behavioral objectives, for programming (also for k-12 schools but different kind).

9. Continuing as a college teacher Arizona State U, 1960-1967. Behavioral department started by A. Staats (and Hudson Jost, dept chair). I. Goldiamond and I were hired in 1960, many other behaviorists (T. Verhave, A. Brownstein, Joel Greenspoon, Lee Meyerson, and others in subsequent years, including F. S. Keller in 1965. • I taught undergrad stat, intro to behavior analysis (using the Holland-Skinner programmed text book) with a rat lab, and several grad courses (advanced stat, verbal beh, exp beh an) I learned about the Keller system from an APA presentation in 1963 and by word of mouth within the behavioral community. Keller came to ASU in 1965 and instituted his PSI version of the intro behavior analysis course, J. G. Sherman a similar one. I wanted to use the Keller system with my courses, and made some attempts. But first, the essential features of the Keller system, as described in the 1968 JABA article.

10. PSI Essential Features: "Goodbye Teacher. . ." (1) The go-at-your-own-pace feature, which permits a student to move through the course at a speed commensurate with his ability and other demands upon his time. (2) The unit-perfection requirement for advance, which lets the student go ahead to new material only after demonstrating mastery of that which preceded. (3) The use of lectures and demonstrations as vehicles of motivation, rather than sources of critical information. (4) The related stress upon the written word in teacher-student communication (with detailed outline, later called study objectives) and, finally: (5) The use of proctors, which permits repeated testing, immediate scoring, almost unavoidable tutoring, and a marked enhancement of the personal-social aspect of the educational process.

11. Attempts at Applying PSI to My Own Teaching When Keller came to ASU (1965) I was responsible for the undergrad stat course, a grad stat course, and others. I taught the grad stat course for 6 students as a PSI course with myself as the only proctor. (Scott Wood was a student.) However there were no resources for a PSI stat course for 40 or so students--the classroom was already assigned, I had only one grad assistant, MWF 50-minute class meetings, etc. However, as with PSI, I emphasized written source material, plus detailed instructions to the student as to what was to be learned (later described as study objectives). Lectures were not used for motivational purposes (this feature of PSI just didn't work), but as with PSI, lectures did not add much new material that was not in the text. I used the smallest unit size possible, which was one week. And I relied on short-answer exam questions.

12. Evolution of a System: Western Michigan U At first I considered my ASU approximation of PSI to be only a compromise with the real PSI which I thought was better. I left ASU in 67 and joined the Psych Dept at WMU: Taught intro stat, undergrad social psych (VB), and some grad courses. At first I tried to add a mastery requirement to the stat course (60 or so students), and I experimented with PSI versions of some of the grad courses. But for class sizes of 40 or more, meeting times other than MWF for 50 min, TTh for 75 min, or once a week for 3 hrs required too much systems management. Some behaviorists at other universities did develop large PSI courses, which however had a number of problems. Some such courses are still functioning (for example the course managed by Ramona Houmanfar at U. Reno ).

13. So I improved the instructions as to how to make use of text and lecture material, which I now called the course study objectives. An example is given a few slides further on. And I also improved on the exam contingencies. Details are given a few slides further on. These improvements permitted me to keep the MWF class meetings and the weekly in-class exams.

14. Our Ph.D. in psych began in the late 70s, and by the early 80s we were offering a two-sequence graduate course (690/691) aimed at preparing our grads to be college teachers. This led to some further refinements of my approach to college teaching. The grade sheet. Example shown. Regrade Request. When a student believes that his or her exam has not been graded accurately he or she may submit a request that the exam be regraded by me. Because the graders grade with a stringency bias, I often give back some points that were taken off, and change the student's grade on the master grade sheet.

15. Remedial Exam. Twice during the semester a special exam is given during a regular class period instead of a lecture. Objectives are provided for this exam. For the Remedial 1, the objectives are those that I consider important for the material that occurred during the first half of the semester. For Remedial 2 the objectives are those that were important during the second half of the semester. The score obtained on the first remedial can replace a lower score obtained during that half of the semester. If the remedial is lower than any scores, it is not used. Similarly for the second remedial. Remedial exams reduce the aversiveness of the relatively strict contingencies of this course--an exam every week, specific study instructions that must be followed, grading on an absolute basis instead of on a curve, no system that permits a student to discard the lowest exam and base the scores on the remainder, no making up for absences except with the remedial exam.

16. The Current System: Study Objectives Students receive a numbered list of 10 to 20 objectives regarding text material for each of the 7 to 11 units of the course. Objectives are numbered so that they can be easily referred to in other objectives and other units (illustrated in next slide). Each objective is given a two or three word name or title. Why? Naming facilitates repertoire organization: To some extent knowledge of a subject matter consists of small chunks, and relations are easier to learn if the chunks have names. Naming also facilitates construction of objectives. Objective names can be listed at the beginning of each unit like a table of contents for the unit, and a table of all the unit objective names can be given at the beginning of the course as a table of contents for the whole course. Each objective has a page and paragraph number, so that the student can easily find the critical material.

17. Sample Unit • Unit 3, Fall, 02, of the intro behavior analysis course (Psych 360) for majors. MWF class meetings, exam every Friday. Objectives had undergone many versions--starting about 15 years earlier. Syllabus statements:

18. Assignment: Basic Principles, IVB10a & b. Rev for Exam 3: U2, Obj 8; Exam 2, Qs 3, 6, 17. 1. Stimulus change decrement defined. The elicitative effect of a CS will show a decrement (will be smaller) if the CS or the ambient environmental stimuli are different in any way from what they were during the original training. This is called stimulus change decrement. FTO: Give this definition. 2. Stim. change decrement demonstration. FTO: With respect to stim. change decrement, be able to exemplify the relevant procedure and the results by writing about a change in an ambient stimulus (IVB10a), and a change in the CS (IVB10b). Remember that in both cases the decrement is a decrease in the response to the CS. Study Figure 1 (page 15) so that you can answer questions about a similar set of graphs on the exam. 3. Stim. generalization defined. The effect of a novel stimulus on . . . 4. Stim. generalization demonstration. Using an example with . . . (and so on with objectives 5 through 10) Here are some sample exam questions for Obj. 1, 2, 3, and 4: (1) Suppose that you have conditioned a dog to salivate when you touch Unit 3: Stim-change decrement and other qualifications 1. S-change decrement def. 2. S-change decrement demo. 3. S generalization def. 4. S generalization demo. 5. Respondent "forgetting." 6. Stim. cond. and rfmt. 7. Rsp-rfmt delay. 8. Delay & rfmt effectiveness. 9. Motivation: Estab. Operations 10. Terminology supplement; Pair vs pairing, stimulus vs. stimuli, reinforcing a stimulus (NOT), responses as rfmt (NOT). Unit 3: Stimulus Change Decrement and Other Qualifications him on the right shoulder. Explain carefully how you would demonstrate stimulus change decrement with respect to an ambient stimulus. Your answer should take the following form: First (a) explain what you would do, then (b) what aspect of the dog's behavior would be the measure of stimulus change decrement. [3] Answer: (a) You have to change one of the stimuli affecting the dog other than the touch on the shoulder (e.g. have the lights in the room dimmer than when conditioning took place) then touch the dog on the shoulder. (b) This should result in less saliva than to the touch when the lights were at their original brightness. (2) Suppose that we condition a man to show vasoconstriction in his right hand when he tastes lemon juice (by pairing the lemon juice taste with putting his hand in cold water). Explain carefully how you could demonstrate

19. Assignmnt: Basic Principles, IVB10a & b. Rev for E3: U2, Obj 8; E2, Qs 3, 6, 17. 1. Stimulus change decrement defined. The elicitative effect of a CS will show a decrement (will be smaller) if the CS or the ambient environmental Ss are different in any way from what they were during original training. This is called stimulus change decrement. FTO: Give this definition. 2. Stim. change decrement demo. FTO: With respect to stim. change decrement, Unit 3: S-change dec and other qualifctns 1. S-change decrement def. 2. S-change decrement demo. 3. S generalization def. 4. S generalization demo. 5. Respondent "forgetting." 6. Stim. cond. and rfmt. 7. Rsp-rfmt delay. 8. Delay & rfmt effectiveness. 9. Motivation: Estab. Opertns 10. Terminology supplement; U3: S-Change Decrement & Other Qualifications be able to exemplify the relevant procedure and the results by writing about a change in an ambient stimulus (IVB10a), and a change in the CS (IVB10b). Remember that in both cases the decrement is a decrease in the response to the CS. Study Figure

20. 3. Stim. generalization defined. The effect of a novel S on . . . 4. Stim. generalization demonstration. Using an example . . . (and so on with objectives 5 through 10) Unit 3: S-Change Decrement & Other Qualifications Here are some sample exam questions for Obj. 1, 2, 3, and 4:(1)Suppose that you have conditioned a dog to salivate when you touch him on the right shoulder. Explain carefully how you would demonstrate stimulus change decrement with respect to an ambient stimulus. Your answer should take the following form: First (a) explain what you would do, then (b) what aspect of the dog's behavior would be the measure of S-change decrement. [3] Answer: (a) You have to change one of the stimuli affecting the dog other than the touch on the shoulder (e.g. have the lights in the room dimmer than when conditioning took place) then touch the dog on the shoulder. (b) This should result in less saliva than to the touch when the lights were at their original brightness. (2) Suppose that we condition a man to show vasoconstriction when he tastes lemon juice (by pairing the lemon juice taste with

21. The Current System: Exams With a MWF class, a 50-minute short-answer essay exam is given on Friday. (With a TTh class the exam is every 3rd class period.) In the first class lecture and in the syllabus, the amount of outside-class study time is stated, based on the 2 for 1 rule (2 hours of outside class study is expected for each hour in class, for a course grade of C (thus for a 3 credit course, 6 hours of outside class study). I use a seating chart and one of the graders makes a record of who was absent from lecture. This is so that I can advise students who are doing poorly regarding their poor attendance, and so that I can use this information if a student comes for help later in the semester, (and help will be correlated with attendance). On Exam days students sit in alternate seats, and the student graders and I walk around the room to make cheating less likely.

22. Students keep their copy of the exam questions when they hand in their answers. Any problems with this? Couldn't students give their exams (and answers) to a friend who takes the course the next semester? Sure, but the detailed study objectives provide more than sufficient information as to what will be on the exams The proportion of objectives covered in an exam is very high, so old exams do not permit any reduced study of new material. They pick up a copy of what I consider to be correct answers to the exam questions as they leave. Students typically spend some time out in the hall as they leave the exam comparing my answers with what they remember that they wrote. My answers are used by graders and myself in the grading process. Exam questions and answers thus become a part of the instructional material for that term and subsequent terms in that I can refer to earlier exams and answers in subsequent assignments.

23. Exam Grading Using undergrad students to help grade the exams requires something other than just letting them do it. Here is the procedure I use. 1. The graders and I each take 12 to 20 exams to grade, which they do at home or some place where they can work alone. (Some instructors grade in their office with the graders.) 2. They have a copy of the same exam answers that the students in the course received when they handed in their exams. 3. And they also have any special grading instructions that I made up as I made up the exam. They also have the general grader instructions (shown on the next slide). 4. They grade using a stringency criterion: If it is not clearly and completely correct, they are to take points off. 5. But most important, I regrade any question on which the student lost points.

24. Grader Instructions 1. We will split the papers between us, 12-20 papers each. (there may be a few items that I grade on all of the exams). 2. If any of the students in the course are friends of yours must not grade their papers--trade them to another grader. 3. Before you start actually taking off points for a particular item, look over several of the papers to see what kind of answers are occurring. This will help you deal with answers that you may wish to give partial credit for. 4. Remember that I will reread all of the answers where you took points off, but I do not generally reread those that you scored as completely correct. Thus you should use a stringency bias. If you are too strict I will have a chance to change your evaluation to make it more in line with my own criteria, but if you are too lenient I will probably not make contact with the student's answer, which would be bad.

25. 5. This is very important: Grade item #1 on all of your papers, then grade #2 on all of the papers, and so on. By the time you have graded several examples of a particular item you will have memorized the necessary information for that item and will grade most accurately. 6. Even though it might seem more efficient, do not grade the items paper by paper. That is, do not grade all of the items on Smith's paper, then all of the items on Jones' paper, and so on. Do it item by item rather than paper by a paper! 7. As you grade the answers and find yourself making judgments of various kinds, make notes so that you can describe any consistent criterion that you used. Then I can rapidly check all of your papers for that item and change them all if I do not agree with your criterion.

26. 8. If it is not likely that the student will be able to see why the answer is incorrect you can write a brief remark that clarifies your basis for taking off points. 9. Do not write anything that can be considered sarcastic, derogatory, critical, etc. If an answer seems completely foolish, resist any temptation to comment on that fact. Do not write "Wow", or place large and repeated question marks or exclamation marks by such answers. 10. Do not award any bonus points. If you find an answer that seems especially clear, tell me about it in your notes to me about the exam. You might say "See Jones' excellent answer for item #11." I will use this information in assigning any bonus points. (See "scoring modification" on the next slide.)

27. 11. Item scoring modification: As I grade an exam I usually find an item or two that had something wrong with it. For example, if an unusual number of students miss the item I may modify the scoring, either reducing its point value or discarding the item. Then I award a 1/2 bonus point for those who did get the item right with the existing scoring. 12. When exams are returned the students receive a a. written commentary on the exam answers b. written explanation of any scoring modifications, c. list of the objectives and exam questions for review as a part of the next unit, d. grade sheet showing their performance on the exam.

28. The Grade Sheet Considering the exam grade (and its relation to the course grade) as the main motivation for studying, it is important for the students to know at all times exactly where they are with respect to this ultimate outcome. In 1980 I started using an Apple 2e computer, and in particular a spread sheet program called VisiCalc (later replace by Excel). From then on I have provided the students in all of my courses with a spread sheet after each exam that shows their exam scores, the scores of the other students, their average score, their projected total if they continue at their present average, and what they need to average for the remainder of the semester to end up with a grade of A, B, C, and D. The grade sheet also shows the average for each exam and a frequency distribution for each exam. (Sample on the next slide.)

29. Exam Return: The Grade Sheet average needed for a grade of projected total student # Exam 1 Exam 2 Exam 3 Exam n Exam 11 average A B C D 1 31 30 29 30 330 34.5 30.0 26.3 22 2 33 363 30.9 26.1 21.2 16.5 32 34 3 27 21 15 np 30.6 25.7 21.0 21 231 4 n 43 average 30.3 30.9 29.1 Total Points=11 X 35 = 385 A=346 B=308 C=269 D=231 33-35 17 20 18 30-32 13 14 17 27-29 6 7 4 24-26 4 0 3 21-23 1 2 0 < 23 2 0 1 N= 41 43 43 np = not possible

30. Additional Course Procedures Regrade Request: When I hand back their graded exams I encourage students to submit a regrade request if they believe that their exams have not been graded accurately. I suggest that they study their exam answers in comparison with my answers, before submitting their requests because some of their regrade requests may be covered by the my answers. Remedial Exam (for absences and to replace a low score): Halfway thru the semester I give out a set of remedial objectives, chosen from the first 5 units. These are objectives that cover main points, such that if a person did well on this exam I would be comfortable replacing their missing score or their lowest score with this score, even though it is not heavily weighted on the particular score that they missed or was low.

31. Summary of Important Features Study units and objectives: Objectives are (a) numbered, (b) named, (c) have page and paragraph location, and (d) often contain a good deal of explanatory material, which is distinguished from the actual objective by the FTO indication. Exams: (a) Students keep a copy of exam questions, (b) receive a copy of my correct answers. Exam grading: (a) Graders and I each grade 12-20 papers, (b) graders use a stringency criterion, (c) I regrade any item on which the student lost points, and (d) item scoring is modified (by me) if responses are unreasonable. When exams are returned: (a) Students receive written commentary on exam answers, (b) explanation of any scoring modifications, (c) units and objectives for review in next unit exam, (d) and a grade sheet which shows their and others' performances on the exam. Additional course procedures: regrade request, remedial exam.

32. Warnings This system is most useful when the instructor wants the student to learn a small number of things, but learn them very well; And where what is learned early is used in later portions of the course--like a statistics course or a behavior analysis course. It is not especially useful when the instructor wants the students to learn a little bit about a lot of topics--survey course. Common errors: Too much material for an intensive course--where the instructor is used to assigning many pages of more than one text, or many difficult journal articles, makes up a large number of very brief objectives--too much for a reasonable exam. Instructor adopts objectives but does not have them closely related to the study objectives (as when the instructor uses the multiple choice exams provide by the text author.)

33. Future Developments If I started anew I would put much of the course material on a website and get rid of paper handouts: Syllabus, Unit objectives, exam answers, exam comments, grade sheet, etc. The students would have to download and print their own copies, so we are not actually eliminating paper, but at least eliminating the cost of the paper to the university. Secure exam site would still be needed. I do not want to use mutliple-choice exams and I do not want the assessment process be open book or a group project.

34. General Results Grade distributions like those for other behaviorally managed courses: 60% with B or better; 83% with C or better, which means that they satisfied the course requirement for a major; 14% will have to retake the course for it to count for their psychology major. Class started with 42 students but 5 withdrew before the midterm without loss of grade points (but will have to pay again to take the course. No problem from the administration re the high grade distribution--no accusation of grade inflation when the amount and type of assignments and the type and amount of assessment is seen. Over the semesters the amount of lecture needed decreased to the point that fairly effective students can get a good grade without attending class except on exam day, and some do.

35. General Results (cont'd.) I did not have to be a systems manager. Supervised two or three grading assistants is all. No special rooms or other facilities. No special schedule arrangements. I spent my time improving objectives, making up exams (questions and answers), grading exams, reacting to the graders graded exams, writing exam comments, updating the grade sheet. Typical Sunday (4 or so hrs.), Thurs (2-3 hrs.) and Fri morning before exam (2-3 hrs.), 3 hrs. actual class time: total = 13 hrs 690-691 results: Grads usually say that the courses were very helpful in their achieving status and praise for their teaching. Very good course evaluations (mine and other faculty using this general approach). Much praise for study objectives, frequent exams, grade sheet, and fairness of grading system.

36. Course Evaluation Comments Alyce Dickinson's Undergrad Course for Majors • Extremely well structured, with immediate feedback. Really helped me mold my study habits. • Never had a professor so thorough and explicit about grades, grading procedures, and handed back graded exams. • Grading was very fair. • The number of exams was great. • Detailed grading sheet encouraged my studying for the exam. • Extremely clear objectives and feedback were very helpful. • Grading procedures and feedback was excellent. • Built a solid repertoire in behavior analysis. • Most beneficial was the way in which the entire class was taught. I wish other classes would take some hints from he Psych dept.

37. Course Evaluation Comments (cont'd.) • I knew what was expected of me before exams so it was the best of any course I have had. • Most beneficial--study objectives and frequency of exams. • Great grading procedure, without explicit study objectives I would not have been as successful in this class. • Exams were very fair. • It was easy to see progress in class and where you stand grade-wise. • I think the way that the class was structured made this class very beneficial and I feel that I learned a lot. • The amount of exams is great because it forces you to keep up with and understand the material. • I have actually learned in this class rather than simply memorizing and regurgitate information on tests.

38. Course Evaluation Comments (cont'd) • Dr. Dickinson has the best teaching methods I have ever encountered. • I thought that the frequent tests would foster more anxiety, but I actually think it was better that way and reduced stress by allowing me to master small amounts before moving forward. • Regarding the grading procedures, they were very clear. I especially liked the spreadsheet so you could map out test scores in the future. • I loved the constant grade sheets. They are extremely helpful. • Regarding what was most beneficial, the grade sheets and the exams back the day after the exam. • (What aspects of the course were most beneficial?) Frequent tests with study objectives and immediate feedback.

39. If you do these things you should be well respected by students and colleagues, and will have many happy years as a college teacher. You may have to cut back on some of the features of this system in order to get some research published, or to apply for funding, but at least you will know what you should be doing if you had the time.

40. THANK YOU FOR YOUR ATTENTION. • If you would like a copy of this slide presentation, or if you would like to ask me anything further about this kind of college teaching contact me by email. jack.michael@wmich.edu