Blended by Design: Designing and Developing a Blended Course

1. Blended by Design: Designing and Developing a Blended Course Jennifer Strickland, PhD, jennifer.strickland@mcmail.maricopa.edu

2. Day 3:Assessment, Student Readiness, Academic Integrity, & Quality Assurance in Blended Learning

3. Objectives • Review options for online assessment techniques • List strategies to assess student readiness for blended learning environments • Describe some ways to prepare for student crisis points • Consider academic integrity and online assessment • Assess quality assurance guidelines used to organize content in an online environment

4. Mapping Your Course, Part III

5. Student Assessment In the good old days…

6. Student Assessment In the networked/information age

7. What’s Changed? CHANGED NOT CHANGED Definitions Institution’s responsibility to inform, educate, and enforce Honor code policiesand procedures Student assessment quality, validity, reliability • Limitless cheating mechanisms • Lines have blurred • Ease of cheating • Ease of monitoring cheating • Ease of preventing cheating

8. What does it look like today? Cheating or Not Cheating?

9. What does it look like today? • Having someone edit students' papers (grammar, style, spelling) • Note-taking services • Sharing password to course management systems • Submitting a paper from a term paper service • Working on a graded assignment together • Revising a paper that was found on the internet • Using a cell phone (IM) to transmit exam information • Looking at another student's work while taking an exam • Sharing computers • Public computers • Other?

10. Cheating and Plagiarism Websites • http://www.cheathouse.com/ • http://www.schoolsucks.com/ • http://www.researchpaper.com/ • http://www.allpapers.com/intro.htm

11. Pedagogical Solutions • Assign work and tests that are due frequently throughout the semester • Assign work that builds sequentially on prior submitted work, such as revisions of drafts • Administer unannounced quizzes or participation • Take-home tests/quizzes

12. Pedagogical Solutions • Require assignment and test responses to relate the subject matter to students' lived experiences or test questions on current events • Meet with students individually online and test/quiz them on course content • Require students to participate in discussion groups • Keep a log and review writing styles of students

13. Pedagogical Solutions • Debrief/interview a student concerning their test/quiz asking specific questions about their answers • Use alternative modes of student assessment such as portfolios, rubrics, self-assessment, peer assessment, and contracts • Use multiple methods of measuring performance • Use application-type exams (PBL, case based learning)

14. Detecting Cheating in Online Environments

15. Alternative Test Methods

16. Integrity TipsAdapted from What Can We Do About Student Cheating? Sally Cole and Elizabeth Kiss. About Campus, May-June 2000 • Define academic integrity as a class • Encourage students to come to you if they are confused about citation practices • Be a good role model; cite sources in your lectures • Talk about academic honesty with your students, and make sure they understand both the reasons and the tools for avoiding plagiarism • Contracts for integrity

17. Principles of Academic Integrity: Changing the CultureAdapted from Donald L. Mc Cabe and Gary Pavela • Affirm the importance of academic integrity—workplace standards. • Encourage student responsibility for academic integrity. • Clarify expectations for students. • Develop diverse forms of assessment. • Reduce opportunities to engage in academic dishonesty. • Challenge academic dishonesty when it occurs and make it public. • Help define and support campus-wide academic integrity standards.

18. Alternative Means of Evaluating Student Performance • Center for Academic Integrity • Assessment and Evaluation for Online Courses • Authentic Assessment Resource Site • Alternative Assessment and Technology (ERIC Digest) • Classroom Assessment Techniques • Virtual Academic Integrity Laboratory

19. Benefits of Assessing Online • Regular feedback • Various types of assessments • Low stakes • Self assessment • Automatic grading & instant feedback • Anytime, any place • Instant access to resources • Can randomly pull certain categories of questions so no two tests are alike • Can build and track portfolio and project based assessments

20. Limits to Online Assessment • Cheating • Quirkiness of technology • Access to online resources & paper mills

21. Suggestions for Better Assessment • Build multiple “check point” assessments • Allow open note/book timed tests • Have in-class quizzes and tests as well • Project-based assessments • Portfolio assessments • Assign group work in a wiki area that tracks student participation

22. Activity • Identify one academic integrity challenge you have or may experience in the future (in a blended environment) • Identify a potential solution • Share

23. CATs: What are they? A method used to inform you on …. • students learning • effectiveness of course content • effectiveness of teaching methods

24. CAT Benefits • Learner-centered • Teacher-prompted • Mutually beneficial • Formative • Fast to administer • Fast to interpret • Non-threatening • Ongoing • Foster trust between student and instructor

25. Choose a learning goal to assess • Choose an assessment technique • Apply the technique • Analyze the data and share the results with students • Respond to the data, i.e., make modifications as necessary Basic CAT Steps

26. 5 Suggestions for CATs • Customize to your specific needs and learning environment (f2f/online) • Should be consistent with your instructional philosophy • Test out a CAT and assess their effectiveness • Allow extra time to carry out and respond to the assessment • Let students know what you learn from their feedback and how you and they can use that information to improve learning

27. CAT Examples • Minute paper • Chain notes • Memory matrix • Directed paraphrasing • One-sentence summary • Exam evaluations • Application cards • Student-generated test questions • Can be easily modified or converted to an online environment

28. Other ideas • Most CATs can be created using a survey tool, or just email your students questions • Polls (polldaddy.com, survey monkey, survey tool in Bb or WebCT) • Drill & practice activities with participation points (softchalk, quizzes) • Peer review & feedback

29. Exercise • Review the CATs and pick one • Identify the goal for your CAT • Develop or adapt an existing CAT for your blended course • Explain why this CAT is helpful/necessary in this particular area of the course • Explain why/where you would use this CAT in a f2f or online environment & is there a pro or a con to moving it online • How and when will students receive feedback on the CAT

30. Assessment Resources • http://honolulu.hawaii.edu/intranet/committees/FacDevCom/guidebk/teachtip/assess-2.htm • http://www.ntlf.com/html/lib/bib/assess.htm • http://www.ntlf.com/html/sf/vc75.htm

31. Break

32. Student Readiness

33. Support in the blended course?

34. Readiness Assessment Strategies Formal Informal Website Welcome materials FAQs Examples Pros/Cons Testimonials • eLearning website • Screening surveys • Pre and post enrollment with feedback • Debunking incorrect impressions • Advisor meeting

35. Assess… • Skills (reading) • Learning styles • Work and study habits • Technical requirements (hardware, software, connectivity) • Need and immediacy for course • Feedback preferences • Ability to self-help (when things are difficult) • Attitude toward the nature of learning online

36. Readiness Means… • Determining who is ready • Ready now = start course • Not ready now = • Tutorials • Support • Advisor meeting • UCF Learning Online

37. Are you ready for MY course? • Take a moment to review some of the course readiness surveys • Identify 2 areas where your students might struggle • Come up with a strategy that you would offer to support students with these • How would you customize these readiness surveys for your course to further communicate to students specific success requirements

38. Student Crisis Points

39. Student Crisis Point Causes

40. What are they? • Moments during your course when students are most likely to need support and assistance • Example: The first time a student logs in to your course web site and cannot successfully locate the address and insert the username and password? • Identify crisis points in advance, so you can make sure that you have a plan in place to mitigate student problems and avoid frustration

41. How do you identify them? • Review the sequence of learning activities and course modules you have planned • What student skills will be required to be successful • Technological skills • Using course management system (tests, finding materials, email, groups, web 2.0 tools, etc.) • Learning skills • Time management • Ability to retain and use your course content

42. Preempting Crisis Points • Identify 4 potential student crisis points • 2 technological • 2 pedagogical • How will you address, support and troubleshoot your students' technological and pedagogical needs during your course

43. Quality Assurance Guidelines and the Blended Learning Environment

44. Quality Matters • Quality Matters Overview and Principles • The Quality Matters Rubric • Quality Matters as a Component of Quality Assurance • Feedback and Input • http://www.qualitymatters.org

45. Course Meets Quality Expectations Course Revision Faculty Course Developers Quality Matters Course Peer Review Process Institutions National Standards & Research Literature Course Rubric Faculty Reviewers Training Peer Course Review Feedback Instructional Designers

46. QM Certified Peer Reviewers • QM-Certified Peer Reviewers • How to interpret the standards (with examples and annotations) • How to evaluate a course (hands-on with sample course) • Reviews are conducted by teams of three peer reviewers • Chair • Peer reviewer (external) • Peer reviewer (SME)

47. What is it anyway? • Quality Matters (QM) is a faculty-centered, peer review process designed to certify the quality of online and hybrid courses and online components • A faculty-driven, collaborative peer review process • Committed to continuous quality improvement • Based in national standards of best practice, the research literature and instructional design principles • Designed to promote student learning and success

48. 700+ faculty trained to review online courses using the rubric • individuals from 158 different institutions in 28 states • More than 2,500 faculty and instructional design staff participated in Quality Matters workshops Quality Assurance

49. The Rubric is the Core of Quality Matters • 40 specific elements across 8 broad areas (general standards) of course quality • Detailed annotations and examples of good practice for all 40 standards • http://www.qualitymatters.org/FIPSE.htm

50. Course Alignment • 5 of the 8 general standards should align: • Course Overview and Introduction • Learning Objectives • Assessment and Measurement • Resources and Materials • Learner Interaction • Course Technology • Learner Support • ADA Compliance Key components must align