Constructed Response Items

1. Constructed Response Items Common Formative Assessment This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

2. AcknowledgementsSpecial thanks to all contributors to the development and revision of this module.The original collection of learning packages was rolled-out for use by Regional Professional Development Center (RPDC) Consultants in July 2013 after being developed by a team of content experts. The collection of learning packages was developed through efforts funded by the Missouri State Personnel Development Grant (SPDG). The following individual/groups are thanked immensely for their hard work in developing this package. ---------------------------------------------------------------------------------------------------------------------------------------- Content Development Support UMKC Institute for Human Development Ronda Jenson, Director Stefanie Lindsay Arden Day Carla Williams Jodi Arnold SPDG Management Team

3. Initial Content Development Team, 2013Rob Gordon, Team Leader, MO PLC Rebecca Rider, SE RPDCAlan Bancroft, C RPDC Kelley Ritter, SW RPDCDebra Cole, StL, RPDC Jay Roth, SW, RPDCCheri Fuemmeler, SE RPDC Jana Scott, HOM, RPDCJane Jackson, NW RPDC Thea Scott, DESEMelanie Whitener Needling, SE RPDC Alicia Wilson, KC RPDCLinda Null, SE RPDC 2016 Revision Team Nancy Steele, Facilitator, NE SIS Deb Drury, NE RPDC Pam Carte, NE RPDC Ginger Henry, DESE Myra Collins, NE RPDC Sarah Spence, C SIS

4. Welcome and Introductions Please take a moment to introduce (or reintroduce) yourself to the group, by telling your name, district, and position. Our trainers for the day are…. 8

6. Collaborative Work Foundations CFA – utilize Quality assessments as feedback to improve teaching & learning. 3 Key Areas 1. Collaborative Teams 2. Common Formative Assessment 3. Data-Based Decision Making DBDM – utilize data for continuous improvement through dialogue about teaching & learning CT – utilize team processes to improve teaching & learning.

7. Common Formative Assessment Modules 4-6 Constructed Response Items Performance Assessments Assessment Items Selected Response Items

9. Missouri Teacher Standards • Common formative assessments address the following Missouri Teacher Standards: • Standard #7: Student Assessment and Data Analysis • Standard #2: Student Learning, Growth and Development • Standard #4: Critical Thinking • Standard #9: Professional Collaboration (Missouri Department of Elementary and Secondary Education, 2013) 13

10. Norms • Start and end on time • Be an engaged participant • Be an active listener – open to new ideas • Use electronics respectfully 14

11. Agenda • Introductions • Formative Assessment Review • Definition of Constructed Response Items • Overview of Constructed Response Items • Overview of Scoring Guides • Critique Constructed Response Items • Identify Higher Level Items 15

12. Learning Outcomes As a result of this training, you will learn… • the difference between closed and open-ended constructed response items. • guidelines for recognizing quality constructed response questions. • to critique scoring guides used with constructed response questions. 16

13. Formative Assessments “[Common assessments are] not standardized tests, but teacher-created, teacher-owned assessments that are collaboratively scored and that provide immediate feedback to students and teachers.” (Reeves, 2012) 17

14. “Thus, it is done by the teacher in the classroom for the explicit purpose of diagnosing where students are in their learning, where gaps in knowledge and understanding exist, and how to help teachers and students improve student learning.” (Perie, Marion, Gong, & Wurtzel, 2007) 18

15. Most formative assessments are small scale – a few seconds or a few minutes and a short cycle. 19

16. What are Constructed Response Items? • Students formulate their own answers. • CR items may be closed or open-ended. SO…. What is the difference between a closed constructed response item and an open-ended constructed response item? 20

17. Constructed Response Items Open-ended CR Items • Measure higher-level cognitive processes such as application, inference, analysis and synthesis • Elicit diverse answers • Mild to high tolerance for diversity • Usually allow for more than one way to arrive at an answer Closed CR Items • Measure knowledge and comprehension • Elicit a right or wrong answer • No tolerance for diversity • Usually allow for only one way to arrive at a correct answer 21

18. Definition “Constructed response Items: Test items requiring students to generate their own answers.” (Popham, 2006a) 22

19. Open-ended Constructed Response Items Students will… • apply knowledge/interpret information. • utilize complex cognitive processes, e.g., inferences. • support or justify answers. • create clear, concise answers. 23

20. Steps in Writing an Open-ended Constructed Response Items • Decide on learning target. • Define mastery. What will students have to know, understand and do to show mastery? • Write the item in clear, concise language. • Develop the scoring guide. • Ask a peer to answer the question to see if the answer given is what you expected. • Revise if necessary. 24

21. Sample Constructed Response Items and Scoring Guides 25

22. Courtesy of NARA #NWDNS-179-WP-1563 The poster above was printed during the Second World War. Why was it printed? How did it try to appeal to women? (National Center for Educational Statistics, 2014) 26

23. Key/Scoring Guide Complete The response answers both parts of the question (Why was it printed? How did it try to appeal to women?) Partial The response either says why the poster was printed, or explains how the poster tried to appeal to women, but does not do both. Inappropriate The response does not link the poster to the circumstances of WWII America, or explain how the poster tried to appeal to women. (National Center for Educational Statistics, 2014) 27

24. Science – Grade 4 Think about where rain comes from and explain why the Earth never runs out of rain. 28 (National Center for Educational Statistics, 2000)

25. Scoring Guide Complete • Student demonstrates understanding of Earth as a closed system by explaining why rainfall is continually produced as part of the water cycle. • Water falls as rain, evaporates and rains again • Water evaporates and then rains from clouds Partial • Student demonstrates understanding of part of the water cycle. • Rain comes from clouds • Water evaporates from lakes, rivers, oceans, etc. • Rain comes from oceans, with no mention of intervening step • Rain is recycled, but no detail provided Unsatisfactory/Incorrect • Student merely restates the question, "because the rain never runs out," or gives an incorrect explanation. 29 (National Center for Educational Statistics, 2000)

26. Critiquing Open-ended Constructive Response Items 30

27. Critique Form for Open-Ended CR ItemsExcept Mathematics • Forces students to demonstrate an in-depth understanding of content • Requires the use of complex cognitive processes or higher level thinking skills • Elicits diverse answers • Allows for multiple points of view and interpretations • Requires support, justification, evidence, and/or explanation of reasoning • Requires the student to refer to specific places in the text that justify their conclusions or claims • Requires students “interfere” or interpret information • Is worded in a clear, concise manner • Prompts the student to do everything required b the rubric for the highest rating • Avoids teaching in the item stem (giving clues in the item stem) • If support is elicited, there is ample information (in story, graph, etc.) for students to use as support • Is written in the simplest language possible 31

28. Critique Form for Higher-Level Math CR Items • The problem allows for multiple solution processes or may be solved in more than one way. (This does not necessarily mean that there are multiple answers; instead there are multiple ways to arrive at the answer). • The problem allows for application of knowledge and/or is set in a real world scenario. • The solution process or how one goes about finding the answer is not apparent. • The problem involves multiple steps. • The problem contains enough information to find a solution and have clarity, but not too much to diminish the problem-solving approach to learning. • The problem is written in such a way to clearly elicit the desired evidence of the student’s problem-solving approach. • When feasible, the problem promotes connections between and among concepts. • When feasible, the problem requires supporting reasoning/data/and/or justification/explanation. • When feasible, the problem allows for conjecture. • The problem accurately assesses the learning target. • The problem is appropriate in terms of grade-level difficulty, cognitive complexity, and reading-level. • The content and terms used are mathematically accurate. • The problem asks students to show a process, communicate a process, and/or explain a problem-solving approach. • The wording used is clear, concise, and to the point. 32

29. Items That Are Not Considered QualitySpeculate the Reasons! • Create a pamphlet to advertise a country. Include the country’s flag, anthem, imports, exports, etc. • Write an essay to explain what the country of Canada is like. • Create a PowerPoint Presentation about the novel Red Badge of Courage. • Draw a chart showing the three branches of government. • Draw a picture of a cell. Be sure to add labels. • Write five events that happened in the story. • Draw a picture of the food pyramid. • Explain the causes of the Civil War. 33

30. Find two major flaws in this open-ended CR item and how might it be revised? In class, we learned that when supply goes down, cost goes up. Explain what would happen to the price of hula-hoops if the largest supplier in the United States went out of business. Use four details to support your answer. 34

31. What are the flaws in these items?Describe how to revise each item. • Did you like the story? Explain why or why not using three details and/or examples from the story. • Explain, what you think will happen to John (the main character) after the end of the story. • Predict what will happen to Mary if she disobeys her mother? Use two details and/or examples from the story in your answer. • What are two historical reasons to explain the present-day appearance of our national flag? 35

32. Stimulus Materials 36

33. Possible Stimulus Materials 37

34. Possible Stimulus Materials 38

35. Locate Stimulus Materials • National Archives • Smithsonian Institute • Marco Polo website • Internet search for charts, graphs, maps, excerpts from documents • Digital video clips from YouTube • Magazine articles • Newspaper articles • Reading A to Z • Project Gutenberg 39

36. Scoring Constructed Response Items 40

37. Scoring a Closed CR Item 41

38. Closed CR Item Name the type of figurative language used in the following sentence: Being famous in Instagram is like being rich in Monopoly. ________________ 42

39. Scoring Guide 1 point Student answer is Simile 0 point Student answer is something else. NR for No Response or B for Blank AB Absent 43

40. Scoring Responses to Open-ended CR Items • Requires a different approach than just grading responses right or wrong. • Open-ended CR questions elicit responses that fall on a wide spectrum of possibilities. • Criterion referenced scoring guide or rubric is used. • Generic or Task Specific =. • Response is rated based in its ability to meet specified criteria. 44

41. Science – Grade 12 • Although nuclear fusion power plants are not currently used for power generation, people want to develop fusion power. Nuclear fusion power plants in the future are expected to solve some of the technological problems of existing nuclear fission power plants. • Describe two advantages nuclear fusion power would have compared with nuclear fission power. (Do not include economic factors.) 45 (National Center for Educational Statistics, 2009)

42. Scoring Guide Complete Answer • Student response provides two correct advantages of fusion power plants compared to fission power plants. Partial Answer • Student response provides one advantage of fusion power plants. Unsatisfactory/Incorrect Answer • Student response is inadequate or incorrect. 46

43. Generic Scoring Guide for an Open-ended CR Math Problem 2 Points: The response contains an acceptable/correct answer and a valid solution process. The response shows an understanding of the process needed to find the answer.   1 Point: The response contains an acceptable/correct answer.  OR   1 Point: The response contains a valid solution process but had minor computational errors. The response shows an understanding of the process needed to find the answer. 0 Points: The response shows severe misunderstanding. 47

44. Open-ended Constructed Response Scoring Guides Scoring Guides for open-ended CR items are often more descriptive. Student answers may be different and support/ examples provided may also be different. Scoring guides need to take these factors into account. 48

45. Advantages and Disadvantages of Different Types of Rubrics (Brookhart, 2013) 49

46. Desired Characteristics of Criteria for Classroom Rubrics (Brookhart, 2013) 50

47. Developing CR Items through Coaching 51

48. Questions and Answers Who wants to ask the first question? 52

49. Reflection Out of all the things you learned about Open-ended CR items, what point stands out as being the “shining star”? 53

50. Closing & Follow-up Common Formative Assessment Constructed Response Items