Item Development Presentation

1. Item Development Presentation June 11, 2012

2. Item Developer’s Tasks • To develop high-quality, standards-based items that will adhere to the Florida Department of Education Item Specification and will be used on three formative district test forms to evaluate students’ strengths and weaknesses on the Next Generation Sunshine State Standards (NGSSS) in Social Studies

3. DOK I DOK II DOK III

4. DOK Sample Question 1 What was the immediate cause of the outbreak of World War I (1914)? A. Formation of secret alliances. B. Competition over Asian and African colonies. C. Accumulation of arms and weapons. D. Assassination of Archduke Ferdinand of Austria.

5. DOK Sample Question 2 Use the table to answer the question. 1860 Which conclusion related to the Civil War can be drawn from the information in this chart? A. The South had more large cities than the North. B. Sectionalism was based on social and economic differences. C. The South was well prepared to win a war against the North. D. Agriculture was no longer important to the United States economy.

6. DOK Sample Question 3 • Which late 19th-century event is portrayed in this cartoon? • economic tariffs on European goods • the incident leading to war with Spain • colonial expansion into Latin America • the capture of the Philippine Islands

7. Cognitive Complexity Levels Recall and Reproduction Skills & Concepts Strategic/ Extended Thinking

8. Guidelines for Writing Multiple Choice Questions

9. Terminology • Item - the entire multiple choice question. • Stem – the first, sentence-like portion of the multiple-choice question. • Alternatives – all possible multiple-choice response options. • Key response – correct answer. • Distracters - the wrong answers. They are called distracters because they should be written to be plausible and, ideally, reflect common misconceptions, thus distracting students with only superfluous knowledge of the tested material.

10. Example What is the process of elimination in a multiple choice question? (stem)   A. Skipping the entire question (distracter)*B. Eliminating the wrong answers (key)  C. Eliminating all extreme answers (distracter)  D. Just guessing (distracter)

11. Example (poor) In World War II the Allied strategy, agreed upon by the U.S. and Great Britain was to • concentrate on defending Japan before turning to Germany. • fight only against Japan, leaving the Russians to fight Germany alone. • take a passive role and limit operations to reacting to Axis moves. • concentrate on defeating Germany first before turning on Japan.

12. Example (poor)Use a direct question (not an incomplete sentence) in the stem In World War II the Allied strategy, agreed upon by the U.S. and Great Britain was to • concentrate on defending Japan before turning to Germany. • fight only against Japan, leaving the Russians to fight Germany alone. • take a passive role and limit operations to reacting to Axis moves. • concentrate on defeating Germany first before turning on Japan.

13. Example (better) What was the Allied strategy, agreed upon by the U.S. and Great Britain, in World War II? • Concentrate on defending Japan before turning to Germany • Fight only against Japan, leaving the Russians to fight Germany alone • Take a passive role and limit operations to reacting to Axis moves • Concentrate on defeating Germany first before turning on Japan

14. Example (poor) Which of the following events abolished the institution of slavery in the United States? • Ratification of the 13th amendment in 1865 • Creation of the Freedmen’s Bureau of 1865 • Compromise of 1850 • Emancipation Proclamation of 1863

15. ExampleArrange numerical alternatives (such as dates) in ascending order. Which of the following events abolished the institution of slavery in the United States? • Ratification of the 13th amendment in 1865 • Creation of the Freedmen’s Bureau of 1865 • Compromise of 1850 • Emancipation Proclamation of 1863

16. Example (better) Which of the following events formally abolished the institution of slavery in the United States? • Compromise of 1850 • Emancipation Proclamation of 1863 • Creation of the Freedmen’s Bureau of 1865 • Ratification of the 13th amendment in 1865

17. Example (poor) Which of the following groups of citizens of the United States were notallowed to cast their votes in elections for President of the United States until the 23rd Amendment was passed?

18. ExampleDo not use negative words, such as “not” or “except” in a stem. Which of the following groups of citizens of the United States were not allowed to cast their votes in elections for President of the United States until the 23rd Amendment was passed?

19. Example (better) Passage of the 23rd Amendment gave presidential election voting privilege to which group of citizens?

20. Example (poor) Which of the following is an example of a success of the “right-to-life” movement ? • It created a constitutional amendment banning abortions. • Achieving restrictions on abortions • Getting the Stewart v. Freeman case to the Supreme Court • All of these choices are correct.

21. ExampleDo not use “all of the above” or “none of the above” Which of the following is an example of a success of the “right-to-life” movement ? • It created a constitutional amendment banning abortions. • Achieving restrictions on abortions • Getting the Stewart v. Freeman case to the Supreme Court • All of these choices are correct.

22. Example (better) Which of the following is an example of a success of the “right-to-life” movement ? • Creating a constitutional amendment banning abortions • Achieving restrictions on abortions • Getting the Stewart v. Freeman case to the Supreme Court • Reversing Roe v. Wade

23. Writing Good ItemsPart I. Emphasis on the Stem • Write questions that examine important concepts; avoid questions addressing minute details. • Make the stem clear and unambiguous. • Use simple grammar and sentence structure. • Do not use vocabulary more advanced than required. • Include in the stem any word(s) that otherwise would be repeated in each alternative.

24. Example (poor) How many grievances against the king were named in the Declaration of Independence?

25. Example How many grievances against the king were named in the Declaration of Independence? Write questions that examine important concepts; avoid questions addressing minute details.

26. Example (better) What major grievance against the king was named in the Declaration of Independence?

27. Example (poor) As the level of fertility approaches its nadir, what is the most likely ramification for the citizenry of a developing nation?

28. Example As the level of fertility approaches its nadir, what is the most likely ramification for the citizenry of a developing nation? Make the stem clear and unambiguous. Do not use vocabulary more advanced than required.

29. Example (better) What would be the likely result of a decline in fertility in a developing nation?

30. Example (poor) I believe that it must be the policy of the United States to support free peoples who are resisting attempted subjugation by armed minorities or outside pressures. This statement was used • to support the Baruch Plan. • to support the Monroe Doctrine. • to support the Good Neighbor Policy. • to support the Truman Doctrine.

31. Example I believe that it must be the policy of the United States to support free peoples who are resisting attempted subjugation by armed minorities or outside pressures. This statement was used • to supportthe Baruch Plan. • to supportthe Monroe Doctrine. • to supportthe Good Neighbor Policy. • to support the Truman Doctrine. Include in the stem any word(s) that otherwise would be repeated in each alternative.

32. Example (better) I believe that it must be the policy of the United States to support free peoples who are resisting attempted subjugation by armed minorities or outside pressures. This statement was used to support which of the following? • The Baruch Plan • The Monroe Doctrine • The Good Neighbor Policy • The Truman Doctrine

33. Writing Good ItemsPart II. Emphasis on Alternatives • Do not let any portion of the stem provide unintended clues. • Do not let the length of alternatives provide unintended clues. • Make all distracters plausible (use common errors). • Make the alternatives mutually exclusive. • Keep the grammar of each alternative consistent. • Be careful to keep items independent, so that the answer to one question does not provide a clue to solving another. • Make the position of the key response (correct answer) random.

34. Example (poor) What was one of the most important domestic economic results of World War II? A. Complete employment for minority Americans B. The end of discrimination for African Americans C. Change in social status for women D. The swift ending of the Great Depression

35. Example What was one of the most important domestic economicresults of World War II? A. Complete employment for minority Americans B. The end of discrimination for African Americans C. Change in social status for women D. The swift ending of the Great Depression Do not let any portion of the stem provide unintended clues.

36. Example (better) What was one of the most important domestic results of World War II? A. Complete employment for minority Americans B. The end of discrimination for African Americans C. Change in social status for women D. The swift ending of the Great Depression

37. Example (poor) “Reaganomics” was based upon which of the following views? A. The government must cut taxes to stimulate investment, which would, in turn, increase employment and tax revenue. B. The government must commit to large expenditures. C. The government must institute wage and price controls. D. The government must increase taxes on corporations and individuals.

38. Example “Reaganomics” was based upon which of the following views? • The government must cut taxes to stimulate investment, which would, in turn, increase employment and tax revenue. B. The government must commit to large expenditures. C. The government must institute wage and price controls. D. The government must increase taxes on corporations and individuals. Do not let the length of alternatives provide unintended clues.

39. Example (better) “Reaganomics” was based upon which of the following views? A. The government must cut taxes to stimulate investment into the economy. B. The government must commit to large expenditures. C. The government must institute wage and price controls. D. The government must increase taxes on corporations and individuals.

40. Example (poor) The English based their claim to territorial rights in North America on the expeditions of which of their explorers? A. Henry Hudson B. Ferdinand Magellan C. Sir Walter Raleigh D. Amerigo Vespucci

41. Example The English based their claim to territorial rights in North America on the expeditions of which of their explorers? A. Henry Hudson B. Ferdinand Magellan C. Sir Walter Raleigh D. Amerigo Vespucci Make all distracters plausible (use common errors).

42. Example (better) The English based their claim to territorial rights in North America on the expeditions of which of their explorers? A. Henry Hudson B. Francis Drake C. Sir Walter Raleigh D. John Rolfe

43. Example (poor) Which of the following was a circumstance of the election of 1789? A. Southern Federalists supported Thomas Jefferson. B. The Federalist Party did not unite behind Adams. C. George Washington took an active role. D. The Federalist party divided over the support of Adams.

44. Example Which of the following was a circumstance of the election of 1789? A. Southern Federalists supported Thomas Jefferson. B.The Federalist Party did not unite behind Adams. C. George Washington took an active role. D. The Federalist party divided over the support of Adams. Make the alternatives mutually exclusive.

45. Example (better) Which of the following was a circumstance of the election of 1789? A. Southern Federalists supported Thomas Jefferson. B. The Federalist Party united behind Adams. C. George Washington took an active role. D. The Federalist party divided over the support of Adams.

46. Example (poor) Why did President Kennedy decide to remove Diem from the presidency of South Vietnam? A. Diem launched attacks on the country's Buddhists. B. Massacring a large number of Viet Cong C. Refusing to allow American soldiers to engage in combat D. Shooting his own brother for treason

47. Example Why did President Kennedy decide to remove Diem from the presidency of South Vietnam? A. Diem launched attacks on the country's Buddhists. B. Massacring a large number of Viet Cong C. Refusing to allow American soldiers to engage in combat D. Shooting his own brother for treason Keep the grammar of each alternative consistent.

48. Example (better) Why did President Kennedy decide to remove Diem from the presidency of South Vietnam? A. Diem launched attacks on the country's Buddhists. B. Diem massacred a large number of Viet Cong. C. Diem refused to allow American soldiers to engage in combat. D. Diem had his own brother shot for treason.

49. Example Which of the following events occurred as a direct result of the Townshend Acts? A. The Committees of Correspondence were formed. B. The First Continental Congress was convened. C. The principle of "salutary neglect" ended. D. The colonists resented mercantilist economic policy. What was the purpose of the colonial Committees of Correspondence created as a result of the Townshend Acts? A. Keep colonial intellectuals in contact with each other B. Publicize grievances against England C. Improve the writing skills of young gentlemen D. Correspond with English radicals who supported the American cause Be careful to keep items independent, so that the answer to one question does not provide a clue to solving another.

50. Example Which of the following events occurred as a direct result of the Townshend Acts? A. The Committees of Correspondence were formed. B. The First Continental Congress was convened. C. The principle of "salutary neglect" ended. D. The colonists resented mercantilist economic policy. What was the purpose of the colonial Committees of Correspondence created in response to the Townshend Acts? A. Keep colonial intellectuals in contact with each other B. Publicize grievances against England C. Improve the writing skills of young gentlemen D. Correspond with English radicals who supported the American cause