1 / 45

Measuring Your Success in Course Redesign

Measuring Your Success in Course Redesign. Sandi Connelly Gosnell School of Life Sciences, RIT, Rochester, NY Sandra.Connelly@rit.edu or RITGenBio@gmail.com. The Dirty Words in Education. Success Efficacy Assessment Evaluations. Our Topics for Today.

rtrisha
Télécharger la présentation

Measuring Your Success in Course Redesign

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Measuring Your Success in Course Redesign Sandi Connelly Gosnell School of Life Sciences, RIT, Rochester, NY Sandra.Connelly@rit.edu or RITGenBio@gmail.com

  2. The Dirty Words in Education • Success • Efficacy • Assessment • Evaluations

  3. Our Topics for Today • How will you measure success? • Will you use historical data (common exams, etc.) to support the efficacy of your assessments (MyLab & Mastering)? • Mastering platform in Sciences similar to MyLab platform in other disciplines • MyLab & Mastering assignments will contribute what percentage to a students final course grade? = assessment • What do you do with the information evaluated?

  4. Today’s Plan • My course structure • Implementation of a few simple techniques • Quick ways to measure success • How to use MyLab / Mastering to maximize student learning • How do other data support MyLab / Mastering use? • How much is too much (or too little!) for a grade item? • How to use MyLab / Mastering for Course Assessment

  5. My Course • General Biology (year-long sequence) • Typical # of students: 250-340 (taught in two sections) • This year 408 • Typical # of different majors: 55 from all RIT Colleges • ~ 50/50 science/non-science • Freshmen -> grad students • Current grade calculation (2013 – 2014): • 65-70% exams; 15% Mastering Bio HW; 10% Mastering Bio quizzes; 5-10% in class participation

  6. Before you measure success…

  7. Are Students Prepared to Succeed? • Preparation (lack of!) comes in many forms • Reading / writing comprehension • English as a Second Language (or 3rd, 4th, ….) • Learning disabilities • Fundamental knowledge in sciences • Study skills • Dedication to the course • Dedication to the degree / college • Etc……

  8. Preparing Students for Success • Homework & Quiz • Assign immediately on Day 1 • Provide timely feedback on the assignment • “Suggestions” from instructor do hold water! • Require regular assignments • In Class Response Activities • Every day! (“mock quiz” type) • Immediate or timely feedback critical

  9. Preparing Students for Success • Be clear about the goals of pre-course activities • E.g. pretests, diagnostic tests, etc. • Benefit to students • Not enrolled in a class in which they cannot succeed • Decreases DWF rates • Increase retention – intangibles • Benefit to you / course • Less time on “catch up”, More time on application • Less stresses associated with DWF, office hours, etc.

  10. So you have “prepared” students … Now what?

  11. Measuring Success – How? Of what? • Retention (D/W/F) • Common exams? • Shared assessments across sections? • Final exam scores • Final course grades • MyLab / Mastering • Alternative learning initiatives

  12. Measuring Success – How? Of what? • What works for … • You • Your students • Your discipline • Your assessment office? • Your supervisor?

  13. Measuring Success – How? Of what? • Let’s focus on MyLab / Mastering as a success tool … • We can add in the other things later …

  14. MyLab / Mastering = X % • 2008-2009: Not much – trial only; assignments optional • 3 attempts per question; unlimited time • 2009-2010: Not much – trial only; assignments optional • 3 attempts per question; unlimited time • 2010-2011: 5% of grade; ~20% never did any; nearly all students who completed the assignments had a 100% avg • 3 attempts per question; unlimited time • 2011-2012: 15% of grade; all complete at least most; avg scores 88-93% • 2 attempts per question; unlimited time

  15. MyLab / Mastering = X % • 2012: 20% of grade HW and 5% on quizzes; all students complete nearly all (> 97% completion rate) • HW = 2 attempts per question; unlimited time • Quiz = 1 attempt per question; 60 min time limit • 2012 Example • Mitosis/Meiosis HW: 96 questions; avg time spent = 44 minutes; avg score = 85.9% • Homework due before lecture • Result? No time in class on process – instead, theory!

  16. MyLab / Mastering = X % • 2013 – 2014: 15% of grade HW and 10% on quizzes; all students complete nearly all (> 98% completion rate) • HW = 2 attempts per question; unlimited time; weekly • Quiz = 1 attempt per question; 60 min time limit; weekly • Food for thought … is this % too much? • Will students cheat, copy, etc.?

  17. Everything has challenges …

  18. What is your biggest challenge? • Time is the untamed obstacle for everyone .. • Take 5 minutes • Sit down! (or better yet – while you are shampooing!) • What information do you really want your students to retain? • WHY? • What do you want your students to leave your class knowing? • WHY?

  19. Challenges to Assessment • Things that I have considered and toyed with … • How much do I want the students to do outside of class on MyLab/Mastering? • Limited to “Time Estimated” = 1 hour • Less if possible! I want their attention!

  20. Challenges to Assessment • Things that I have considered and toyed with … • What is the value of a long assignment if the students don’t do it, copy it from another student, just use all of the hints, etc.? • For students who are failing exams, but have 100%+ on the homeworks, I look at their time spent in MyLab / Mastering • Discuss issue with them in person and make note in the Institute Reporting

  21. Challenges to Assessment • Things that I have considered and toyed with … • Are the students just memorizing answers, or are they applying the information and retaining it? • Have stopped assigning reading / test bank questions • Implement more low stakes quizzes • Forced repeated recall of information in class

  22. You have data … Now what?

  23. Student Data Assessment • Why assess? • My general interest • Are the students learning what I think they are learning? • How can I improve this course? • University call to assess Gen Ed courses (Accreditation) • Assessment needs? • My interest • Exam and homework scores before and after MyLab / Mastering implementation • University Guidelines: 1 – 4 scale of comprehension

  24. Student Data Assessment • What was assessed? • Kept it simple to start! • Exam averages • Add as I have interest / time! • Participation in MyLab / Mastering • Scores on MyLab / Mastering • Time spent on MyLab / Mastering

  25. Student Data Assessment • Exam grades (presented herein) • All stored in ExcelTM since 2007 • Mastering (not presented) • Export grades & time spent from Mastering to ExcelTM • Processing, Compilation, and Statistical Analyses • ExcelTM

  26. Exam Score Trends Err = St Err Avg n = 244

  27. Talk to me data! • Higher exam averages every year – but leveling out • Significant differences only in plants – but trends are obvious • Too many students from too many majors to see huge jumps in scores despite all efforts • Tightening of exam scores • Standard deviations and ranges decreasing • Increasing amount of “average time spent” on MyLab / Mastering assignments

  28. Assessment Challenges • Why do exam grades change? • Things I have considered • What changes from year to year other than MyLab / Mastering? • Students? • Accessibility of tutors? • Me? • Order of topics covered? • Lab instructors?

  29. Assessment Challenges • Why do exam grades change? • Things that I want to consider • Students • Time spent on studying (other than MyLab / Mastering)? • Major? • College entrance exams? AP Bio? • DWF rates by major and biology “experience” • Me • Lectures improved? • Lecture structure change with online assignments?

  30. “Napkin Notes” • General observations: • Increased time spent on assignments after 1st exam of term • Increased # questions in office hours on assignments • Increased time spent “off topic” in lecture – not a bad thing! • Changes implemented to date: • Lecture time is more interactive • Participation activities with the students (worksheets, group work, think-pair-share, flipped lectures, etc.) • Modifications to assignments and quizzes to “challenge” the students in preparation for class and exams

  31. Advice for Instructors Looking to Assess Success • Don’t assess while you are teaching the course! • Assessment can be done any time! • Store the data, save it for later • Even if you have time … subconsciously not good for your teaching! • Don’t try to assess everything at once! • Pick one exam in your course, and compare the average ± standard deviation (standard error) between years • Even simpler? • Compare exam averages between class sections. TA DA! You just did an assessment!

  32. Advice for Instructors Looking to Assess Success • Don’t expect MyLab / Mastering to make your students in to geniuses! • These are tools! And how the students implement the tools (accept the tools!) will be highly variable. • You may not see immediate changes in exam grades • Accept the “intangible” as a result! • You can not grade student enthusiasm (which means you can’t assess it – sorry!) • Student enthusiasm is contagious – not just to other students – but also to you!

  33. Advice for Instructors Looking to Assess Success • What can you do with your data? • FLAUNT IT! • White papers • Peer-reviewed papers • Teaching portfolio • Promotion dossier • Assessment office

  34. Final thoughts …

  35. What else is assessable? • I got my class back! • Not spending class time on giving quizzes, answering questions about homework format, etc. • What do we do then? Talk about Biology! • AND … most of my assessment is built in to MyLab / Mastering and my course management system! • Don’t reinvent the wheel! • Download grades, assign your assessment strategy, .. TA DA!

  36. What else is assessable? • HUGE time savings in my classes • I have “recovered” 35 minutes per week in class (of 150 min) with no quizzes and homework things in class • Prepared students means < 45 min lecture! > 105 min biology exploration! • Time saving is so huge … how huge is it? • Enough that I am completely “flipping” one section • DATA! DATA! DATA! DATA! DATA! DATA! DATA! DATA!

  37. But .. The Most Important Thing to Me?

  38. What the Students Have to Say! • "Mastering Bio helps a lot! The more you have us do, the more I learn!” • "I wish that our exams were as interactive as Mastering! I love learning that way!" • "(Favorite part about Mastering) aspects of the assignments helped me to better grasp the materials through the videos. I would not have done as well on those parts of the test without Mastering.” • "More Mastering - 'nuf said!"

  39. Let’s Recap • How will YOU measure success? • Will YOU use historical data (common exams, etc.) to support the efficacy of your assessments (MyLab & Mastering)? • MyLab & Mastering assignments will contribute what percentage to YOUR students final course grade? = assessment • What do YOU do with the information evaluated?

  40. The Dirty Words in Education • Success • Efficacy • Assessment • Evaluations

  41. THOUGHTS!? Sandi Connelly Gosnell School of Life Sciences, RIT, Rochester, NY Sandra.Connelly@rit.edu or RITGenBio@gmail.com

More Related