Continuous course improvement through feedback - experiences and challenges

1. Continuous course improvement through feedback - experiences and challenges Karin Väyrynen Tietojenkäsittelytieteiden laitos Business Process Modeling

2. Background of Business Process Modeling 130 enroled 101 started to participate actively 97 finished exercises -> continuous course improvement by collecting feedback throughout the course First implementation of the course Each lecture contained at least 2 activating lecture tasks Mandatory for all 2nd year students at TOL No experience on actual workload of exercise tasks Opintori 2012, Karin Väyrynen

3. Feedback collection in the BPM course Feedback concerning a certain lecture/exercise Feedback concerning course in general Feedback throughout the course Weboodi written feedback L6 L7 L4 L1 L2 L3 L5 L8 L9 E1 E2 E3 E4 oral feedback Discussion List in Optima Anonymous Feedback Box in Optima … and the quality of the assignments! Opintori 2012, Karin Väyrynen

4. Lecture task • Discuss for 1 min in teams of 2-3 What challenges have you encountered when collecting feedback? Opintori 2012, Karin Väyrynen

5. Challenges • Motivating students to give feedback • Explain why you collect feedback • Whether and how to react to offensive or unjustified feedback? • Conflicting feedback • When and how to collect feedback • During or after the lecture? • Anonymous or with names? • Keeping the big picture in mind Opintori 2012, Karin Väyrynen

6. Lessons learned Collect feedback Students are willing to give feedback again Evaluate feedback Students see their opinion matters Make changes based on feedback visible • Emphasize students’ responsibility • Explain how different feedback channels should be used • Reserve enough time for • collecting feedback • responding to feedback • Continuous feedback cycle Opintori 2012, Karin Väyrynen

7. Collecting continuous feedback can be frustrating … …and students will ALWAYS find something to complain about … Opintori 2012, Karin Väyrynen

8. … but it really seems to be worth the effort!  Kiitos! Pidin kurssista :) Ja näit todella paljon vaivaa kurssin eteen. Great Job!! I think this was one of the best courses I have attended. The way the course was held helped me to learn things better. For once a teacher who actually seemed to like what she was doing. Your enthusiasm towards the topic was inspiring. Keep up the good work and continue smiling :) I hope that there will be more courses which will be held like this. Opintori 2012, Karin Väyrynen

9. Summary of feedback collection methods in course Business Process Modeling and Examples of Feedback I received

10. Summary: Feedback collection in the course • Feedback concerning a certain lecture • Feedback forms after lecture concerning that lecture (3 times) • Feedback brainstorming session (students discuss a topic in small teams) • Oral feedback after lecture / exercise meeting • Feedback possibility during whole duration of the course • Discussion list in Optima • Anonymous feedback box in Optima • Overall course feedback • Feedback form after final lecture • Same feedback form in Optima open also after the final lecture • Feedback in WebOodi • Other ways of getting feedback • The models the students created in the exercises • Each student will return an analysis of his/her own work- and learning-process in the course (feedback on whether my time estimates were OK)

11. Examples of feedback received in lecture feedback form after L1 – things that hindered learning In the way the lecture was held In your own behavior Forgot to take a notebook for paper notes (not usually needed at TOL) Being tired Could have slept more last night Well… my attitude about the laptop use  Sometimes my thoughts wander I was too anxious to answer to any of the question, even though I might have known the answer Hard to maintain 100% concentration on less than 8 hours of sleep Coming late • The no laptop rule • Some questions I didn’t hear or they came so fast • A bit slower please, taking notes slows me down a bit. Also a few more examples would be nice • Too much arguing of how to organize this course • Some questions I didn’t hear or they came so fast • Muuten hyvin, mutta itse olen tottunut hakemaan/katsomaan luentokalvot tietokoneelta. Toivoisin, että yli 20-vuotiaat yliopisto-opiskelijat saisivat itse päättää miten/miksi opiskelevat

12. Examples of feedback received in lecture feedback form after L1 – things that supported your learning In the way the lecture was held In your own behavior Concentrating on what the lecturer actually says I always keep an open mind Writing notes I focused more to lesson than usual because couldn’t use my laptop I was listening all the time. Maybe because my laptop was in my backpack Being generally acceptive to studying Putting away my laptop  High level of interest Listened carefully, paid attention A cup of coffee • I’ve never realized how distracting other people’s laptops really are • The lecture itself seems really good • It helps that the language is English • Energinen luennoitsija joka kiinnostunut aiheesta, youtube video • The whole time lecture was engaging. It felt like there was interaction with the audience • Discussions and interactivity • In some way, the strictness (kuri) of the lecture is quite welcomed. It keeps me focused • Hyvät, havainnolliset esimerkit • Good presentation

13. Examples of feedback received in anonymous feedback box • I just want to thank you for the enthusiastic attitude and the very informative emails you are sending. You are doing very good job at informing students. But, I was wondering could it be possible to include like TL;DR(Too long, Didn't read) in your emails? Since some people are really busy and they don't have time to read long emails. They just want to get the information fast and short. It would be really helpful • Too much group work.. Impossible to find time to work as a group and separating into subteams does not help, we still have know how to use the other tool and compare them and write the analysis.. Possible and fair within these time limits? Don't think so. • Considering the laptop ban...I'm sure many will write about this, but i will too. I, for example, read all the lecture material from my laptop, since i can't quite see all the way to the screen. And no, i can not come closer to the screen, because i get headaches from staring upwards for 2 hours. Not that it matters, but the lecturer can't even really ban laptops anyway, all it does is simply makes one look bad in the eyes of students. • There should be some kind of list of points in Optima, where you could check out your own point sum! Also, If you have up-to-date list of lecture participation, it would be great to have it here in Optima also.. This kind of arrangement was used in 'Johdatus tutkimustyöhön' -course, and it was a great way to follow your own progress during the course.

14. Examples of changes made based on feedback Up-to-date list of lecture attendances in Optima Instructions in pdf format in Optima instead of doc Decreased length requirements for one exercise analysis report (the students really had too little time to do the report) Creating workspace for student teams in Optima Adding a brief summary of the most important things in my information emails to the students (my emails are too long and contain too much information) Give additional instructions about exercise tasks BPM 2012, Karin Väyrynen