New Research in the Pedagogy of Learning with Clickers

1. New Research in the Pedagogy of Learning with Clickers Stefanie Mollborn Angel Hoekstra University of Colorado at Boulder

2. Acknowledgements • We thank the President’s Teaching and Learning Collaborative at the University of Colorado for their support of our research.

3. Learning goals • Goals-driven pedagogy • What do you want your students to have learned after taking one of your courses? • Brainstorm

4. Learning goals in sociology • Attending class and participating • Understanding/applying concepts and theories • Critically evaluating concepts and theories • Using them to understand the life experiences of self and others • Other goals in other disciplines

5. How to make students’ learning “stick”? • Problems with retention of material in traditional lectures, listening fatigue • Active learning as a solution • Practical issues with getting students to attend, keep up with readings, and participate in large classes

6. “Clickers” (audience response systems) • Increasingly popular as a way to increase active learning, monitor attendance, motivate reading • Audio and visual stimulation • Emerged in the natural sciences • Flexible pedagogical tool, but they are not themselves a pedagogical strategy—you have to do that

7. Sample question Have you taught using clickers in the past? A. No B. Yes, and I’ve used i>Clickers C. Yes, and I’ve only used other types of clickers (e.g. infrared) D. Don’t know/other

8. Sample question • If you are considering trying clickers in your classroom, what is your most important concern about using them? A. I am already using clickers or don’t want to use them B. I’m worried about technical issues C. I’m worried about how to use them to achieve my goals for the class D. I’m worried that they will make my preparations too time-consuming E. Other/don’t know

9. Prevalent clicker strategy • ConcepTests (Mazur, 1997) to encourage problem-based learning • Proven pedagogical strategy • Usually includes points for correct answers • Points for trying can also function as attendance tracking

10. What would happen to the seasons if the earth’s tilt was increased from 23 degrees to 40 degrees? A.Seasons would be less extreme. B. Seasons would be pretty much the same as now C. Seasons would become much more extreme. D. None of the above

11. What would happen to the seasons if the earth’s tilt was increased from 23 degrees to 40 degrees? A.Seasons would be less extreme. B. Seasons would be pretty much the same as now. C. Seasons would become much more extreme. D. None of the above

12. Amending clicker use to address other learning goals • Moving beyond ConcepTest paradigm of understanding and applying ideas • Encouraging critical thinking • Students as data points: past experience of self and others • Encouraging reading comprehension and participation in whole-class discussion

13. Reading quiz question • What gender combination leads to the lowest likelihood of negotiating, as well as a poor evaluation if the candidate does negotiate? A. Female evaluator-female candidate B. Female evaluator-male candidate C. Male evaluator-female candidate D. Male evaluator-male candidate E. There were no differences

14. Concept test • Does the “sex labeling” of occupations affect supply-side gender segregation, demand-side gender segregation, or both? A. Supply side B. Demand side C. Both D. Neither E. Don’t know/other

15. Critical thinking question • How much do you think cultural factors explain the differences in violent behaviors that we see between men and women? A. Not much at all B. A little C. They are sometimes useful D. They explain most of what we see E. Don’t know/other

16. Critical thinking question • How well do you think England’s findings match your own and/or your friends’ sexual experiences in college? A. I’m female: Pretty/very well B. I’m female: Not very/not at all well C. I’m male: Pretty/very well D. I’m male: Not very/not at all well E. Don’t know/don’t want to answer

17. Past experience question • When you were growing up, which of your parents earned the most money? A. Don’t have 2 parents/one or both don’t work/varied from year to year B. Dad usually earned a lot more C. Dad usually earned a little more D. Mom usually earned a little more E. Mom usually earned a lot more

18. Demographic question • What is your race? A. White B. Black or African American C. American Indian or Alaska Native D. Asian or Pacific Islander E. Other or multiple categories

19. 2000 U.S. Census data

20. Comprehension question • How well do you feel you understand the reading assignment due today? A. Very well—don’t go over it more B. Pretty well, but wouldn’t mind more C. Just OK—I’d like more review D. I’m pretty lost and need help E. I didn’t read the assignment

21. Students’ own questions • Can be an assignment or an open invitation • Submitted “on the fly” or ahead of time • Learning to design hypotheses, figure out what empirical information would test them, design an instrument to collect that information

22. Putting it all together • 10-minute chunks of lecture • Questions with small-group discussions • Show histogram, then large-group discussions • Other active learning activities: chalkboard lists, small group work • Questions form transitions between mini-lectures

23. Our study • How do students perceive that this combination of clicker techniques affects their class experiences and learning? • 6 sections of 3 sociology courses • 385 surveys, 456 free writes, 10 interviews, 100 pages of field notes • Angel has even more data from other disciplines

24. General findings • Most students have past experience using clickers, many already own one • A majority approve of this pedagogical strategy of using clicker questions for diverse purposes • Strong positive effect on attendance, but some backlash • Complicated effect on motivating students to keep up with readings

25. General findings • All of the students interviewed said that if they were the instructor, they would keep using clickers this way

26. Findings on question types • Love/hate relationship with reading quiz questions • Most find other question types (concept tests, critical thinking, and past experience questions) to be useful • Many enjoy getting immediate feedback on their own comprehension of readings and others’ opinions

27. Student quotes • “I think the clickers are very helpful in helping me to understand the material. I like how they stop and force me to think about the material that has just been presented to me, instead of just writing aimlessly throughout class.”

28. Student quotes • “I enjoy the clickers. It saves time and helps [with] participation. For some students, speaking or being called on in class is uncomfortable … Clickers enable every student to participate equally without being singled out.”

29. Student quotes • “It is cool to see your own experience in relation to the rest of the class. It allows us to gauge the differences between each other and opens our eyes.” (e.g., gender questions)

30. Other question types to address other learning goals? • Brainstorm

31. Important issues to consider when using clickers, based on student assessments

32. Classroom environment • Importance of explaining what you’re doing and why • Effects of a right/wrong answer vs. no right answer focus • Importance of encouraging discussions

33. Confidentiality and ethics • Benefits of asking confidential questions • Effects in the classroom • What you can and can’t promise • Ethical boundaries on what to ask

34. Conclusions • Clickers are a pedagogical tool, and it’s up to you to find a good pedagogical strategy • A variety of question types can address different learning goals • Can make students feel like they’re engaged and learning in large lectures

35. Thank you! Questions? Stefanie Mollborn mollborn@colorado.edu Angel Hoekstra angel.hoekstra@colorado.edu Slides and handout will be posted at: http://www.colorado.edu/ibs/hb/mollborn