2. Web-Based Quizzes/Surveys for Meaningful Learner Assessment

1. 2. Web-Based Quizzes/Surveys for Meaningful Learner Assessment • Dr. Curtis J. Bonk • Indiana University and CourseShare • http://php.indiana.edu/~cjbonk • cjbonk@indiana.edu

2. Online StudentAssessment

3. Assessment Takes Center Stage in Online Learning(Dan Carnevale, April 13, 2001, Chronicle of Higher Education) “One difference between assessment in classrooms and in distance education is that distance-education programs are largely geared toward students who are already in the workforce, which often involves learning by doing.”

4. Focus of Assessment? • Basic Knowledge, Concepts, Ideas • Higher-Order Thinking Skills, Problem Solving, Communication, Teamwork • Both of Above!!! • Other…

5. Assessments Possible • Online Portfolios of Work • Discussion/Forum Participation • Online Mentoring • Weekly Reflections • Tasks Attempted or Completed, Usage, etc.

6. More Possible Assessments • Quizzes and Tests • Peer Feedback and Responsiveness • Cases and Problems • Group Work • Web Resource Explorations & Evaluations

7. Measures of Student Success(Focus groups, interviews, observations, surveys, exams, records) • Positive Feedback, Recommendations • Increased Comprehension, Achievement • High Retention in Program • Completion Rates or Course Attrition • Jobs Obtained, Internships • Enrollment Trends for Next Semester

8. Student Basic Quantitative • Grades, Achievement • Number of Posts • Participated • Computer Log Activity—peak usage, messages/day, time of task or in system • Attitude Surveys

9. Student High-End Success • Message complexity, depth, interactivity, q’ing • Collaboration skills • Problem finding/solving and critical thinking • Challenging and debating others • Case-based reasoning, critical thinking measures • Portfolios, performances, PBL activities

10. Richness Coherence Elaboration Relevancy Timeliness Completeness Persuasiveness Originality Insightful Clear/Logical Original Learning Fdback/Responsive Format Thorough Reflective Overall Holistic Sample Portfolio Scoring Dimensions(10 pts each)(see: http://php.indiana.edu/~cjbonk/p250syla.htm)

11. E-Peer Evaluation Form Peer Evaluation. Name: ____________________ Rate on Scale of 1 (low) to 5 (high): ___ 1. Insight: creative, offers analogies/examples, relationships drawn, useful ideas and connections, fosters growth. ___ 2. Helpful/Positive: prompt feedback, encouraging, informative, makes suggestions & advice, finds, shares info. ___ 3. Valuable Team Member: dependable, links group members, there for group, leader, participator, pushes group. ___ Total Recommended Contribution Pts (out of 15)

12. E-Case Analysis Evaluation Peer Feedback Criteria (1 pt per item; 5 pts/peer feedback) (a) Provides additional points that may have been missed. (b) Corrects a concept, asks for clarification where needed, debates issues, disagrees & explains why. (c) Ties concepts to another situation or refers to the text or coursepack. (d) Offer valuable insight based on personal experience. (e) Overall constructive feedback.

13. Issues to Consider… • Bonus pts for participation? • Peer evaluation of work? • Assess improvement? • Is it timed? Allow retakes if lose connection? How many retakes? • Give unlimited time to complete?

14. Issues to Consider… • Cheating? Is it really that student? • Authenticity? • Negotiating tasks and criteria? • How measure competency? • How do you demonstrate learning online?

15. Online Testing Tools

16. Pause…What tools do you use or know about?

17. What can Online Tests Do? • Assess student progress • Allow for self-assessment • Provide standards for success • Timed testing and retesting • Opportunity for instructor commenting

19. Using WebCT Quizzes in a High-Demand Environment(Brothen & Wambach, Technology Source, May/June 2003) • “For several years we have taught our psychology courses with the Personalized System of Instruction (PSI; Keller, 1968), a highly researched version of the mastery learning teaching method. Students in PSI work at their own pace to read a textbook with direction from a study guide and, when they are ready, take chapter quizzes; after they master one chapter, they move on to the next.”

20. Using WebCT Quizzes in a High-Demand Environment(Brothen & Wambach, Technology Source, May/June 2003) • “Several reviews and meta-analyses (Keller, 1974; Kulik, Kulik, & Bangert-Drowns, 1990; Kulik, Kulik, & Cohen, 1979; Robin, 1976; Ryan, 1974) have found superior student learning in PSI compared to traditional lecture/discussion methods. For example, in the 26 college psychology courses analyzed by Kulik, Kulik, and Bangert-Drowns, the effect size in favor of PSI over traditional instruction was 0.71.”

26. Test Selection Criteria (Hezel, 1999) • Easy to Configure Items and Test • Handle Symbols • Scheduling of Feedback (immediate?) • Provides Clear Input of Dates for Exam • Easy to Pick Items for Randomizing • Randomize Answers Within a Question • Weighting of Answer Options

27. More Test Selection Criteria • Recording of Multiple Submissions • Timed Tests • Comprehensive Statistics • Summarize in Portfolio and/or Gradebook • Confirmation of Test Submission

28. More Test Selection Criteria(Perry & Colon, 2001) • Supports multiple items types—multiple choice, true-false, essay, keyword • Can easily modify or delete items • Incorporate graphic or audio elements? • Control over number of times students can submit an activity or test • Provides feedback for each response

29. Learner-Content Interactions

31. More Option 6

33. More Option 6

34. More Option 6

35. (see Thiagi.com Or deepfun.com) Puzzle games Solve puzzle against timer Learn concepts Compete Get points Online Fun and Games

36. Students Play Online Jeopardy Game www.km-solutions.biz/caa/quiz.zip

37. More Test Selection Criteria(Perry & Colon, 2001) • Flexible scoring—score first, last, or average submission • Flexible reporting—by individual or by item and cross tabulations. • Outputs data for further analysis • Provides item analysis statistics (e.g., Test Item Frequency Distributions).

38. Web Resources on Assessment • http://www.indiana.edu/~best/ • http://www.indiana.edu/~best/best_suggested_links.shtml • http://www.indiana.edu/~best/samsung/ Rubric for evaluation technology projects: http://www.indiana.edu/~tickit/learningcenter/rubric.htm

39. Online Survey Tools for Assessment

40. Perhaps you have heard of my survey techniques?

41. Sample Survey Tools • Zoomerang (http://www.zoomerang.com) • IOTA Solutions (http://www.iotasolutions.com) • QuestionMark(http://www.questionmark.com/home.html) • Survey Solutions from Perseus (http://www.perseusdevelopment.com/fromsurv.htm) • Infopoll (http://www.infopoll.com)

42. Sample Survey Tools • Active Feedback • (http://www.activefeedback.com/af) • SurveyKey • (http://www.surveykey.com) • EZSurvey from Raosoft • (http://www.raosoft.com/) • SurveyShare (http://SurveyShare.com; from Courseshare.com)

46. Electronic Voting and Polling 1. Ask students to vote on issue before class (anonymously or send directly to the instructor) 2. Instructor pulls our minority pt of view 3. Discuss with majority pt of view 4. Repoll students after class (Note: Delphi or Timed Disclosure Technique: anomymous input till a due date and then post results and reconsider until consensus Rick Kulp, IBM, 1999)

47. Poll Students Asynchronously Web Conferencing Tools (e.g., Sitescape Forum)

48. Poll Students Synchronously Web Conferencing Tools (e.g., HorizonLive)

49. Survey Student Opinions (e.g., InfoPoll, SurveySolutions, Zoomerang, SurveyShare.com)