1 / 68

Chapter 16

Evaluating Knowledge. Chapter 16. Chapter 16 Outline. Purposes of Knowledge Testing Levels of Knowledge Types of Knowledge Tests Construction Administration and Scoring Analysis and Revision Questionnaires. Purposes of Knowledge Testing. Assigning a grade (summative evaluation)

salvador-henson
Télécharger la présentation

Chapter 16

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evaluating Knowledge Chapter 16

  2. Chapter 16 Outline • Purposes of Knowledge Testing • Levels of Knowledge • Types of Knowledge Tests • Construction • Administration and Scoring • Analysis and Revision • Questionnaires

  3. Purposes of Knowledge Testing • Assigning a grade (summative evaluation) • Measuring progress (formative evaluation) • Providing feedback to students or participants • Motivating students or participants • Documenting effectiveness • Assessing instructional effectiveness

  4. Levels of Knowledge • Understanding can range from superficial to thorough. • Bloom’s taxonomy of educational objectives • Knowledge • Comprehension • Application • Analysis • Synthesis • Evaluation

  5. Educational Testing Service • Uses three levels of knowledge for Education Field Tests: • Knowledge • Comprehension • Application

  6. Characteristics of a GoodKnowledge Test • Fair • Covers the course content • Uses appropriate test format • Clear and understandable (no tricks) • Reliable and valid

  7. Students’ Pet Peeves • Trick questions • Poorly-written items • No opportunity for feedback • Too easy/too difficult • floor/ceiling effect • Testing outside the content boundaries • Too much rote memorization of facts with limited relevance

  8. Types of Knowledge Tests • Essay vs. Objective • Mastery vs. Discrimination

  9. Essay Test • People answer each item with whatever information they choose and write answers in sentences. • Quick to construct, but time consuming to score.

  10. Objective Test • Potential answers are provided with each test item. • true-false • multiple choice • matching • Time consuming to construct, but quick to score.

  11. Mastery Test • Used to determine whether students have mastered the material. • Items test information everyone is expected to know. • Formative evaluation with criterion-referenced standards (e.g., graded pass-fail). • Performance standards tend to be high. • e.g., 80% to 90% correct for passing.

  12. Discrimination Test • Purpose is to differentiate among students. • Summative evaluation with norm-referenced standards. • Items are written to discriminate among different knowledge levels. • Items should come from higher levels of Bloom’s taxonomy. • Because discrimination tests are more difficult than mastery tests, performance standards should be lower.

  13. Construction ofKnowledge Tests • Advantages of instructor-made tests: • tend to cover material stressed • terminology is consistent with teaching • tend not to include irrelevant material • tend not to omit important material

  14. Steps in Constructing a Knowledge Test • Construct a table of specifications • define content area • Decide the nature of the test • choose types of item to be used • Construct the test items • Determine test format and administrative details

  15. Table of Specifications - the Blueprint • Content validity evidence • What is the content domain? • Relevance (what does/does not belong in the content domain?) • Representativeness (what is the appropriate balance of content?) • Determined by importance, or time spent teaching that content.

  16. Example Table of Specifications

  17. Types of Test Items Closed-ended (“objective”) • correct answer is provided • more difficult to construct, easier to grade • do not test partial knowledge • may reward guessing Types include: • true/false • multiple choice • matching

  18. Types of Items Open-ended (“subjective”) • student provides correct answer • no knowledge, no score • rewards partial knowledge (may seem fairer) • easier to construct, more difficult to grade Types include: • short answer • fill in the blank • essay

  19. Item Construction Guidelines • Determine the knowledge, understanding, abilities • Determine the most appropriate format

  20. Multiple choice items • Terms: stem, foil, distractor • Use complete statement, question, or instruction in stem • Test only one concept at a time • Provide plausible distractors • Numerical answers should be in order, and not overlap each other • Avoid “compound” answers, unless the material dictates it

  21. Multiple choice items • Avoid negatives, and especially double (and triple!) negatives • Ensure that stem and foils are all grammatically compatible, similar in length and “parallel” in content • Keep to factually or logically supportable correct answers (avoid opinion) • Could a good student answer without the foils? • Avoid “irrelevant cues” – e.g., always, never, common terms between stem and correct answer

  22. Common Errors • All distractors are not plausible. • Item is ambiguous. • Wording the correct response more precisely than the distractors. • Specific determiners. • Irrelevant clues. • Grammatical clues

  23. Common Errors • Look at the following multiple-choice items • Can you figure out the answer? • What is wrong with the item? • How could each item be improved?

  24. Question 1: In Cornwall, a tourist or holidaymaker is called an: • rubberneck • emmet • pastie • tin mine

  25. Question 1: In Cornwall, a tourist or holidaymaker is called an: • rubberneck • emmet* • pastie • tin mine

  26. Question 1: In Cornwall, what term is used to describe a tourist or holidaymaker? • rubberneck • emmet • pastie • tin mine

  27. Question 2: What is the best way to get from Buckingham Palace to Elephant and Castle? • in the Queen’s carriage • with an umbrella • in a straight line • via Piccadilly Circus

  28. Question 2: What is the best way to get from Buckingham Palace to Elephant and Castle? • in the Queen’s carriage • with an umbrella • in a straight line • via Piccadilly Circus*

  29. Question 2: What is the best way to get from Buckingham Palace to Elephant and Castle? • in the Queen’s carriage* • with an umbrella* • in a straight line* • via Piccadilly Circus*

  30. Question 2: Of the following routes, which one is the most direct route from Buckingham Palace to Elephant and Castle on the London Underground? • via Sloane Square • via Marble Arch • via Paddington Station • via Piccadilly Circus

  31. Question 3: Use of ID cards has led to a decrease in crowd violence at which one of the following venues? • Tesco’s supermarkets • Elton John concerts • Soccer matches • The Natural History Museum

  32. Question 3: Use of ID cards has led to a decrease in crowd violence at which one of the following venues? • Tesco’s supermarkets • Elton John concerts • Soccer matches* • The Natural History Museum

  33. Question 3: Use of ID cards has led to a decrease in crowd violence at which one of the following professional sporting venues? • Rugby matches • Cricket matches • Soccer matches* • Tennis matches

  34. Question 4: Which famous British knight of the realm starred in the movie “Bridge Over the River Kwai”? • Duke of Wellington • Lord Louis Mountbatten • Earl Grey • Sir Alec Guinness

  35. Question 4: Which famous British knight of the realm starred in the movie “Bridge Over the River Kwai”? • Duke of Wellington • Lord Louis Mountbatten • Earl Grey • Sir Alec Guinness*

  36. Question 4: Which famous British knight of the realm starred in the movie “Bridge Over the River Kwai”? • Sir Laurence Olivier • Sir John Gielgud • Sir Anthony Hopkins • Sir Alec Guinness*

  37. Question 5: At what age can children leave the British school system? • 15 years • 16 years • 17 years • 18 years

  38. Question 5: At what age can children leave the British school system? • 15 years* • 16 years • 17 years • 18 years

  39. Question 5: At what age can children leave the British school system? • 15 years* • 16 years* • 17 years* • 18 years*

  40. Question 5: What is the minimum age at which children are legally allowed to leave the British school system? • 15 years* • 16 years • 17 years • 18 years

  41. Question 6: What is the nickname for the professional basketball team of Birmingham (England)? • Rangers • Bullets • Celtics • Gunners

  42. Question 6: What is the nickname for the professional basketball team of Birmingham (England)? • Rangers • Bullets* • Celtics • Gunners

  43. Question 6: In which British city is the Bullets professional basketball team located? • Birmingham • Bradford • Bolton • Barnsley

  44. Administration and Scoring • Provide comfortable, distraction-free testing site. • Consider test security: • all students should face same direction • students should sit far enough apart to inhibit cheating • use alternate forms of a test when necessary

  45. Before the Test • Prepare students. • Eliminate test-wise advantage: • grammatical clues • specific determiners • Proofread before photocopying test.

  46. During the Test • Help students pace themselves. • Answer individual questions carefully. • Control cheating: • Negates reliability and validity of test. • Negative attitudes toward those who cheat. • Negative attitudes toward teacher who does not control cheating. • Negative attitude toward testing in general. • Control the environment.

  47. After the Test • Correct and report test scores as quickly as possible. • Report scores anonymously. • Do not misuse or misinterpret test scores.

  48. Analysis and Revision • Overall Difficulty • Mean • Variability • Standard deviation. Larger S, more the test discriminates. • Reliability • KR20 or KR2

  49. Analysis and Revision • Difficulty of each item. • Discrimination or validity of each item. • Quality of each response. • In a multiple choice test, some students should select each response.

  50. Kuder-Richardson (KR) • Estimate of internal consistency reliability by determining how all items on a test relate to the total test. • KR formulas 20 and 21 are typically used to estimate Rxx of knowledge tests. • Used with dichotomous items (scored as right or wrong). • KR20 = coefficient alpha

More Related