Mastering Multiple-Choice

1. Mastering Multiple-Choice Make sure you understand the structure of a multiple-choice question and some basic guidelines to follow when creating multiple-choice tests!

2. Multiple-Choice Question Breakdown • The typical multiple choice question has 3 parts: • Stem: short text that defines the problem, setting, and provides pertinent information to solve the problem. • Question: Usually positioned right after the stem. • Answer Options: The range of choices students can select from- amongst them are distracters.

3. Example of Stems and Questions • In the equation 5x+4 = 20, solve for x. • The term “osteoporosis” refers to: The stem phrases the question, the question lets the learner know what to look for in terms of the answer choices.

4. Answer Options • Answer options are the range of choices the learners can select from to correctly solve the question. • In addition to the 1 correct answer, there are a couple of distracter choices meant to discriminate those who know the content of the test from those who do not.

5. Guidelines for Crafting Multiple Choice Tests • The following slides are useful for teachers to keep in mind while sitting down to develop test questions…

6. 1. When to Write a Test • NEVER sit down and attempt to write an effective test in one day! • Higher Level Thinking Questions take a long time to develop! • A good strategy is writing a few questions after each class session • This way it is simply a matter of assembling the questions into an exam when needed. • In addition, it will better reflect the content you taught in class!

7. 2. All Questions Need to Stand Alone • Craft questions that are completely independent! • This means avoid creating questions that depend on knowing the answers to other questions on the test! • This is a common complaint of students.

8. 3. High Level Learning Questions • Focus on constructing higher order thinking questions that are going to test how well students master concepts. • Too often teachers fall into the trap of creating questions that merely ask for recalling information! • These questions DO NOT test overall understanding of knowledge, rather they test students’ abilities to remember basic facts and statistics.

9. 4. Clarity is Key! • Always write the stem first. • Always phrase the stem as clear as possible. • According to students, one of the biggest issues with multiple-choice tests deal with unclear questions! • Unclear questions are deemed as confusing and discriminate against students who know the material.

10. 5. Hold the Words Please! • Avoid using extra language in the stem other than what is absolutely necessary. • “Padding” stems with surplus words only takes MORE time for the teacher to construct, and MORE time for students to sort through the question.

11. 6. Repetitive Language • It is good practice to include any language in the stem that you would have to repeat in each answer option. • This saves the teacher time when designing the test! • This saves the student time when taking the test as they do not need to waste test time re-reading questions and answer choices.

12. 7. Positive Questions = Positive Responses • It is better for a student to read a positively phrased question, than a negatively phrased question. • Negative questions are often misread or misinterpreted. • If you must use a negative stem, be sure to emphasize the negative words and never use double negatives.

13. 8. Correct Answers • The best strategy for teachers to employ is to write the correct answer choice immediately after writing the stem. • This is a way for the teacher to ensure that it is the best response and that it answers the question accordingly. • In the case of questions that ask to “select the best answer,” using this strategy makes sure that the correct answer choice is appropriate.

14. 9. Don’t Give the Answer Away! • When viewing answer options, students can usually guess the right answer based on the structure of the question! • To ensure this does not happen, write incorrect options to match the correct answer in terms of length, complexity, phrasing, and style.

15. 10. Think Smart • Just as you will write the correct answer choice after phrasing the stem, you should write all distracters after writing the correct answer choice. • This makes the teacher focus on creating plausible distracters.

16. 11. Make it Believable • Create distracters that are realistic. • Try including extra information within the choices that could be seen as possible due to common fallacies about the content.

17. 12. What??? • Avoid using language that is completely foreign to your students. • This could help students narrow down answer choices based on words they do not know. • This discriminates against students who know the content but who are confused by the question structure.

18. 13. Diminutive Distinctions • Avoid making answer options where there are only slightly fine distinctions between answers, unless such distinctions are important in the course. • This is a basic rule to try to abide by because students may get confused figuring out how the answer options differ.

19. 14. Distinct Distracters • Create distinctive distracters! • If all of the distracters are similar in content, style, and appearance, students will catch on and figure out they are distracters! • Use words that should be familiar to students in the distracters to differentiate from those who know the material and test-wise students.

20. 15. Suspicious Syntax • Avoid writing verbal clues that give away the answer! • Watch out for grammar, syntax, words that appear in the stem of the question or correct answer response, and stating correct options in “textbook” language. • Students are trained to look for these errors and capitalize on them!

21. 16. “Degrees of Right” • Be really careful creating questions with terms like “All” “Never” and “Always,” unless it is a specific fact that you are testing. • There will always be students who read into the question too much and get penalized because of the varying “degrees of right.” • In addition, avoid using “All of the Above” or “Both A and B” • This common error makes it easier for students to guess with only partial knowledge!

22. 17. None of the Above…No Way! • Be extremely cautious when using “None of the Above.” • This term should ONLY be used if it is testing absolute answers (i.e. Math tests) • If utilizing “None of the Above,” it should be the correct answer response 25% of the time in 4 answer option tests.

23. 18. Randomize Responses! • Be sure to randomize the position of correct answer responses! • Students tend to pick up on patterns of letters on scantrons! • You can do this by alphabetizing answer options or using random numbering.

24. Recap • Are you starting to realize that there are many ways for students to “strategize” taking multiple-choice tests? • This is why a large portion of students love these “multiple-guess” tests, because they profit off of the construction errors their teachers run into! • It is extremely important to craft test questions that make it difficult for students to guess and get the question right. • Utilizing high-level thinking questions, and following the guidelines provided can ensure that students have to know the content for the test!

25. The people of the United States A continent located in the Northern Hemisphere. Are extremely interested in reality television. Are descendants from countries all over the world. A land that houses many different cultures and religions. Mini Exercise: What’s Wrong with This Question? Check out the grammatical errors! A and D can be completely eliminated because they don’t flow with the stem and question. B works, but it is doubtful a teacher would test on that! C is the logical choice! You did not even need to know the material! It is VITAL to construct clear, and grammatically correct questions!

26. Mini Exercise: What’s Wrong with This Question? Which of the following cities houses Disney? • Orlando, FL • Anaheim, CA • Miami, FL • San Francisco, CA Can you guess what is wrong with this question? It is extremely vague and not specific! Disney what? World? Land? There are 2 possible answers! The other 2 distracters are so unrealistic you can eliminate them immediately. Do you see the importance of phrasing clear, specific questions with plausible distracters?

27. Examples of Good Questions • Why do farmers rotate their crops from year to year? • To kill insects that eat the crops. • To conserve the nutrients in the soil. • To confuse animals that prey upon crops. • To make marketing the crops easier. Notice how all of the answer options are about the same length and are composed in the same style/phrasing. All choices seem plausible.

28. Examples of Good Questions • Which of the following best illustrates the law of gravity? • Mike fills his glass with water. When he puts too much ice in the glass, the water spills over the brim. • Joe’s fast car is able to travel a greater distance in a shorter amount of time than Steve’s bicycle. • At a fireworks show, you will see the flash of the explosions before you hear the actual noise. • A kicker punts a football high into the air and watches the ball falls 10 yards short of the goal post. All answer options are in phrased in similar ways, and are similar in length and complexity. Here the student needs to understand the law and how it is applied! Higher Level Thinking!

29. Feeling Confident! • You are now armed with the tools to write effective, high-level thinking objectives! • You are now aware of how multiple-choice questions are structured, and some guidelines to follow to construct effective questions! • Are you ready to move on and learn about different types of multiple-choice questions? If you are, please continue to Presentation 5. If you want to review, please do so and move on only when ready.