1 / 81

These Confusing Technology Times: Making Decisions About What to Assess and Evaluate

These Confusing Technology Times: Making Decisions About What to Assess and Evaluate. Dr. Curtis J. Bonk Indiana University and CourseShare.com http://php.indiana.edu/~cjbonk cjbonk@indiana.edu. Confusion Reigns. How allocate time? When to assess? How to assess? How grade teamwork?

vincent-warner
Télécharger la présentation

These Confusing Technology Times: Making Decisions About What to Assess and Evaluate

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. These Confusing Technology Times: Making Decisions About What to Assess and Evaluate • Dr. Curtis J. Bonk • Indiana University and CourseShare.com • http://php.indiana.edu/~cjbonk • cjbonk@indiana.edu

  2. Confusion Reigns • How allocate time? • When to assess? • How to assess? • How grade teamwork? • Whose work is it?

  3. Other Issues? • …

  4. Student Assessment:Product Focus

  5. Traditional Assessment Methods • Most often, students are assessed in one of the following knowledge-focused ways • Objective test questions • Essay test questions • Papers/Reports • Projects • All are product-oriented in nature

  6. Most Assessment Tools • Focus on tests • Automatic grading/feedback • Test pools • Timing • Favor objective questions • Few tools to facilitate other forms of assessment • File exchange/dropbox

  7. Focus of Assessment? • Basic Knowledge, Concepts, Ideas • Higher-Order Thinking Skills, Problem Solving, Communication, Teamwork • Both of Above!!! • Other…

  8. Technology Assessments Possible • Online Portfolios of Work • Discussion/Forum Participation • Online Mentoring • Weekly Reflections • Tasks Attempted or Completed, Usage, etc.

  9. Richness Coherence Elaboration Relevancy Timeliness Completeness Persuasiveness Originality Insightful Clear/Logical Original Learning Fdback/Responsive Format Thorough Reflective Overall Holistic Sample Portfolio Scoring Dimensions(10 pts each)(see: http://php.indiana.edu/~cjbonk/p250syla.htm)

  10. More Possible Assessments • Quizzes and Tests • Peer Feedback and Responsiveness • Cases and Problems • Group Work • Web Resource Explorations & Evaluations

  11. E-Case Analysis Evaluation Peer Feedback Criteria (1 pt per item; 5 pts/peer feedback) (a) Provides additional points that may have been missed. (b) Corrects a concept, asks for clarification where needed, debates issues, disagrees & explains why. (c) Ties concepts to another situation or refers to the text or coursepack. (d) Offer valuable insight based on personal experience. (e) Overall constructive feedback.

  12. Possible Methods of Assessment • Review of online group work spaces • Evidence of regular and substantial contributions • Self and peer assessment • Have students rate team members on various dimensions • Have students indicate where work plan was followed/not followed • Student reflection • Have students write brief reflections on their group process, indicating what they might change the next time

  13. E-Peer Evaluation Form Peer Evaluation. Name: ____________________ Rate on Scale of 1 (low) to 5 (high): ___ 1. Insight: creative, offers analogies/examples, relationships drawn, useful ideas and connections, fosters growth. ___ 2. Helpful/Positive: prompt feedback, encouraging, informative, makes suggestions & advice, finds, shares info. ___ 3. Valuable Team Member: dependable, links group members, there for group, leader, participator, pushes group. ___ Total Rec’d Contribution Pts(out of 15)

  14. Assessment Issues

  15. Issues to Consider… • Bonus pts for participation? • Peer evaluation of work? • Assess improvement? • Is it timed? Give unlimited time to complete? • Allow retakes if lose connection? How many retakes?

  16. Issues to Consider… • Cheating? Is it really that student? • Authenticity? • Negotiate tasks and criteria? • How measure competency? • How do you demonstrate learning online?

  17. Catching da cheaters!

  18. Increasing Cheating Online($7-$30/page, http://www.syllabus.com/ January, 2002, Phillip Long, Plagiarism: IT-Enabled Tools for Deceit?) • http://www.academictermpapers.com/ • http://www.termpapers-on-file.com/ • http://www.nocheaters.com/ • http://www.cheathouse.com/uk/index.html • http://www.realpapers.com/ • http://www.pinkmonkey.com/ (“you’ll never buy Cliffnotes again”)

  19. Reducing Cheating Online • Ask yourself, why are they cheating? • Do they value the assignment? • Are tasks relevant and challenging? • What happens to the task after submitted—reused, woven in, posted? • Due at end of term? Real audience? • Look at pedagogy b4 calling plagiarism police!

  20. Reducing Cheating Online • Proctored exams • Vary items in exam • Make course too hard to cheat • Try Plagiarism.com ($300) • Use mastery learning for some tasks • Random selection of items for item pool • Use test passwords, rely on IP# screening • Assign collaborative tasks

  21. Reducing Cheating Online($7-$30/page, http://www.syllabus.com/ January, 2002, Phillip Long, Plagiarism: IT-Enabled Tools for Deceit?) • http://www.plagiarism.org/ (resource) • http://www.turnitin.com/ (software, $100, free 30 day demo/trial) • http://www.canexus.com/ (software; essay verification engine, $19.95) • http://www.plagiserve.com/ (free database of 70,000 student term papers & cliff notes) • http://www.academicintegrity.org/ (assoc.) • http://sja.ucdavis.edu/avoid.htm (guide)

  22. Turnitin Testimonials "Many of my students believe that if they do not submit their essays, I will not discover their plagiarism. I will often type a paragraph or two of their work in myself if I suspect plagiarism. Every time, there was a "hit." Many students were successful plagiarists in high school. A service like this is needed to teach them that such practices are no longer acceptable and certainly not ethical!”

  23. Online Assessment Concerns • Problem: Cheating on tests • Copying from neighbor • Copying from course materials • Someone else taking the test • Problem: Plagiarism • Submitting someone else’s paper (previous class) • Copying from (online) sources • Buying paper online • Both are product-oriented concerns

  24. Assessment: Process Focus(Vanessa Dennen, Sept 2002)

  25. Assessing Process • Easy to do • Many technology tools will archive student work/interactions • Students create a document trail in process • Helps students develop metacognitive knowledge • Instructors structure/model/encourage productive work processes • Students learn how to manage their own work processes

  26. Why Assess Process? • For the instructor … • Provides formative feedback on course (e.g., helps gather data about why students have difficulty with product-oriented assessments) • Provides sense of instructor guidance • Clarifies who is doing most work in small group assignments • Helps prevent cheating

  27. Why Assess Process? • For the student … • Typically improves the quality of their products • Helps them develop productive work processes • Puts on a schedule • Shows that you care about individual growth

  28. Assessment Project Cycle • From Classroom Assessment Techniques by Angelo & Cross (1993) • Step 1: Plan • Choose class • Focus on assessable question • Design project to answer question

  29. Assessment Project Cycle [2] • Step 2: Implement • Teach target lesson • Collect assessment data • Analyze data • Step 3: • Interpret results • Communicate results • Evaluate assessment project

  30. I. Term Papers • How to do online: • Have students each start their own thread and post topic of interest • Peers and instructors give feedback • Students post thesis statements, research sources, etc., with iterations of feedback • Final paper is posted

  31. Term Paper Assessments • Product: the paper • Process: quality and timeliness of student work from time when paper is assigned • Process: quality and timeliness of feedback provided to peers • Process: responsiveness to feedback received from instructor and peers

  32. II. Discussion Assignments 1. Chain of thought • Have students develop a solution to a problem • Have students indicate what led them to a particular conclusion, method or approach • Can be done in a discussion board

  33. Discussion Assignments 2. Theory to Practice • Have students match up theories you are learning about to actual problems • Present students with problems and have them explain what theories they would use to solve these problems and how they would approach it • Debrief the assignment

  34. Discussion Assignment 3. Synthesizer • Have students take roles being the weekly synthesizer of class discussion • Add a “meta” level in which students narrate their own experiences while reading the weekly discussion

  35. III. Group Projects • Tools used • Chat: brainstorming ideas, making group decisions, regular way to feel connected (should be archived) • Discussion board: commenting on drafts • E-mail: quick feedback • File exchange: sharing project files • MS Word: Track changes • HINT: If you don’t have a tool that will work, refer students to yahoo groups: http://www.groups.yahoo.com

  36. Group Project Assessments • Product: project files that are turned in • Process: online archive demonstrating • Who contributed what • Who provided peer feedback • Who worked in a timely manner • How collaborative a group was • Process: peer ratings • Process: interim instructor consultations

  37. III. Project Assignments 1. Work Plans • Have students develop a plan of work for their project • Make them outline topic, schedule, resources needed, division of labor and anticipated form of final deliverables • At end of project, have students evaluate how well they followed their own plan and how useful it was

  38. Project Assignments 2. Research Trail • Have students document the steps they took in the research process and the results • Ask for a brief reflection on how effective their process was and what they might change the next time

  39. Project Assignments 3. Process Presentations • Have students focus on their process as well as their product in class presentations • To maintain focus, ask them to share 3 main lessons learned • Might ask for some process documents to be shared, like an early draft

  40. Project Assignments 4. Design Journal • Have students maintain a journal of all ideas related to their project • Encourage sketches, lists, organizational charts, etc. • Require journals to be turned in with final projects

  41. IV. Reflection Assignments • Have students keep a weekly journal of their thoughts on readings and course content AND real-world related instances that they noticed • May make these public, with each student having their own discussion thread

  42. Making it Happen • Learners need to see that process is valuable: • Model appropriate processes • Provide students with scaffolding (guide sheets) to structure their processes • Give students feedback on their process • Require students to reflect on their processes • Grade students on process

  43. Online Testing Tools

  44. Choice: Select companies that specialize in online assessment.

  45. Or: Use what the courseware package gives ya…

  46. Test Selection Criteria (Hezel, 1999) • Easy to Configure Items and Test • Handle Symbols • Scheduling of Feedback (immediate?) • Provides feedback for each response • Randomize Answers Within a Question • Weighting of Answer Options • Supports multiple items types: multiple choice, true-false, essay, keyword

  47. More Test Selection Criteria • Recording of Multiple Submissions • Comprehensive Statistics • Summarize in Portfolio and Gradebook • Confirmation of Test Submission • Incorp graphic or audio elements? • Timed Tests

More Related