Connecting Summative Assessment to Improving Teaching and Learning

1. Connecting Summative Assessment to Improving Teaching and Learning May and June 2016 California http://tinyurl.com/h7d4hub/

2. Workshop Goals Participants will learn: • the rightful place and purpose of the summative assessments • to use results to inform the instructional process by connecting curriculum, instruction and assessment • to access supplementary resources and reference materials to assist local training • to use tools to reflect on current practice to improve teaching and learning • a variety of ways to display data to communicate a particular message

3. The Many Faces of Testing

4. Overview of Major Assessment Types http://www.csai-online.org/sites/default/files/resources/4666/CSAI_AssessmentTypes.pdf

5. Rightful Place/Purpose: Assessment Frequency and Impact on Instruction Statewide Summative Classroom Formative

6. A Balanced Assessment System Summative assessments benchmarked to college and careerreadiness (Grades 3–8 and Grade 11) Teachers and schools have information and tools to improve teaching and learning Standards set expectations on path to college- and career- readiness All students graduate college- andcareer-ready Digital Library Formative assessment tools and practices for teachers to improve instruction Interim assessments Flexible, open, and used for actionable feedback

7. Statewide summative assessments are like icebergs—it pays to pay attention

8. Rightful Place, Rightful Purpose for Statewide Summative Assessment • Provide a general direction—we must dig deeper to determine cause • Focus on groups, programs, and disaggregation • Rarely provide definitive answers, but raise many questions, allowing reflection on context and practice • Provide an entry point into a collaborative, honest conversation • Provide a necessary story

9. Using Summative Results • Comprehensive, complex, and difficult process • Not a checklist • Not meant to be completed at a single staff meeting or professional development event • Focus on improving learning • Not solely about increasing scores • Collaborative process that requires: • Honesty • Willingness to commit the time • Ability to handle ambiguity • Patience

10. Data Examination Is Complicated

11. . . . about the rightful place/purpose of summative assessment results • How do you currently use statewide summative assessment results in the curriculum, instruction, and assessment planning process within your LEA? • What is the place and purpose of the summative assessment in your LEA right now?

12. The Smarter Balanced Hierarchy ofItem Development and Reporting of Scores Evidence-Centered Design Overall Claims Content Claims Targets Evidence Statements ITEMS

13. Concepts of Evidence-Centered Design 1. Define the domain 2. Define claims to be made 3. Define assessment targets 4. Define evidence required 5.Develop items or performance tasks

14. Relationships Between Common Core Standards, California Frameworks, and Assessments English Language Arts/Literacy Content Specifications (Claims) English Language Arts and Literacy in History/Social Studies, Science, and Technical Subjects English Language Arts/English Language Development Framework

15. ELA/Literacy Content Specifications (Grade 7 ELA Example) http://www.smarterbalanced.org/assessments/development/

16. Relationships Between Common Core Standards, California Frameworks, and Assessments English Language Arts/Literacy Content Specifications (Claims) English Language Arts/Literacy Item Specifications (Claims and targets per claim) English Language Arts and Literacy in History/Social Studies, Science, and Technical Subjects English Language Arts/English Language Development Framework

17. Example of Item Specifications (Abbreviated) Evidence Required for Target 1 The student will identify text evidence (explicit details and/or implicit information) to support a GIVEN inference or conclusion based on the text. http://www.smarterbalanced.org/assessments/development/

18. Relationships Between Common Core Standards, California Frameworks, and Assessments English Language Arts/Literacy Content Specifications (Claims) English Language Arts/Literacy Item Specifications (Claims and targets per claim) English Language Arts and Literacy in History/Social Studies, Science, and Technical Subjects English Language Arts/English Language Development Framework English Language Arts/Literacy Assessment Blueprint

19. Sample ELA/Literacy Summative Assessment Blueprint http://www.smarterbalanced.org/assessments/development/

20. Relationships Between Common Core Standards, California Frameworks, and Assessments English Language Arts and Literacy in History/Social Studies, Science, and Technical Subjects English Language Arts/Literacy Content Specifications (Claims) English Language Arts/English Language Development Framework English Language Arts/Literacy Item Specifications (Claims and targets per claim) English Language Arts/Literacy Summative Assessment English Language Arts/Literacy Assessment Blueprint Mathematics Content Specifications (Claims) Mathematics Item Specifications (Claims and targets per claim) Mathematics Framework for California Public Schools Mathematics Mathematics Summative Assessment Mathematics Assessment Blueprint

21. Building a Logical Argument Common Core State Standards Claim Assessment Target Evidence Student Response

22. . . . about the test design and the connections to the instructional cycle • To what extent do the teachers at your school or LEA understand how the test is constructed and how it is connected to the standards and the frameworks? How does this connection have implications for instructional planning?

23. Review of Available Summative Scores

24. Available Summative Assessment Scores • Scale scores • Achievement levels • not met, nearly met, met, exceeded • Claim levels • below, near, above • Target scores • worse, similar, better • insufficient information

25. Scale Score – Achievement Level Relationship – Grade 3 Example

26. Smarter Balanced ELAThreshold Scale Scores http://www.cde.ca.gov/ta/tg/ca/sbelathresholdscre.asp

27. Low, Medium and High Bands (Divide the difference by three = 21 pts) (Divide the difference by three = 21 pts) 2367 to 2387 2388 to 2408 2409 to 2431 2432 to 2450 2451 to 2469 2470 to 2489 (Divide the difference by three = 21 pts) (Divide the difference by three = 21 pts) 2416 to 2433 2434 to 2452 2453 to 2472 2473 to 2491 2492 to 2511 2512 to 2532

28. Low, Medium and High Bands (cont.) 2015 2432 2016 2470

29. Comparing Year to Year http://www.cde.ca.gov/ta/tg/ca/caasppssrinfo.asp

30. Claim Scores: Example of a Claim Claims are broad, evidence-based statements about what students know and can do as demonstrated by their performance on the assessments. For example: English Language Arts/Literacy Claim #1 Students can read closely and analytically to comprehend a range of increasingly complex literary and informational texts.

31. Use of Claim Scores • Reported at an individual and aggregate level • Reported for all claims in ELA and mathematics

32. Assessment Targets • Assessment targets connect the content standards to evidence that will be collected from the assessment. • Targets map the Common Core State Standards (CCSS) onto assessment evidence that is required to support the content categories and claims. • Targets are used to guide the development of items and tasks that will measure the CCSS.

33. Example of an Assessment Target (Grade 7 ELA Claim 1 Example) Target 1 Key Details: Identify explicit textual evidence to support inferences made or conclusions drawn.

34. Use of Target Scores • Reported at an aggregate level relative to group performance • Reported for all claims in ELA, but only for Claim 1 in mathematics • Only targets with 10 or more items in the item pool for a grade are included in target reports http://www.caaspp.org/rsc/pdfs/CAASPP.target-report-FAQs.2016.pdf

35. What’s one thing you will take away from this quick review of the scores? • What additional information do you need? . . . about the summative assessment scores

36. Tools for Analysis of the Smarter Balanced Summative Results

37. Supporting Documentation • Content Specifications • Item Specifications • Blueprints • Achievement Level Descriptors (ALDs) • Policy, range, threshold, and reporting • Claim Descriptors • Score Reports • Additional Data

38. Achievement Level Descriptors • Text descriptions of the knowledge, skills, and processes demonstrated by students at each level. • Four types of levels or categories of performance • Policy and Content • Range • Threshold • Reporting http://www.smarterbalanced.org/assessments/scores/

39. Base for the ALDs (page xvi)

40. Range ALDs • Composed of grade- and content-specific descriptors • Describe the cognitive and content rigor that is encompassed within particular achievement levels • Describe the knowledge, skills, and processes expected of students • Used by test developers to guide item writing

41. Range ALDs (Grade 7 ELA Claim 1 Example) Level 1 Level 2 Level 3 Level 4 http://www.smarterbalanced.org/assessments/scores/

42. Range ALDs (cont.)(Grades 6-8 Math Claim 2, 4 Example) Level 1 Level 2 Level 3 Level 4 http://www.smarterbalanced.org/assessments/scores/

43. Relationship Between Range and Threshold Descriptors Students performing at Level 1 should be able to identify textual evidence that minimallysupports a basic idea drawn about texts of low complexity. Students performing at Level 4 should be able to citestrong and thorough textual evidence to support a complex inference, analysis… Students performing at Level 3 should be able to citesufficientand relevant textual evidence that adequately supports a complex inference… Students performing at Level 2 should be able to citerelevant textual evidence to support a simple inference, analysis, interpretation, or conclusion … Level 1 Level 2 Level 3 Level 4 The student who just enters Level 3 should be able to… The student who just enters Level 4 should be able to… The student who just enters Level 2 should be able to… Adapted from Figure 1 in the Initial Achievement Levels Descriptors document.

44. Threshold ALDs • Areused to guide standard setting • Define the minimum performance required for meeting a particular achievement-level expectation • Reflect the knowledge, skills, and processes that are expected of students • Note: • The knowledge, skills, and processes in ALDs are cumulative. • The student who has achieved the threshold Level 3 is assumed to have the knowledge, skills, and processes of the range Levels 1 and Level 2 ALDs.

45. Threshold ALDs (Grade 7 ELA Claim 1 Example) Level 1 Level 2 Level 3 Level 4 http://www.smarterbalanced.org/assessments/scores/

46. Threshold ALDs (Grade 6-8 Math Claim 2, 4 Example) Level 1 Level 2 Level 3 Level 4 http://www.smarterbalanced.org/assessments/scores/

47. Start thinking about how you might weave these documents into planned professional learning, instructional planning, and local assessment development in the coming school year… . . . about the documentswe’ve just reviewed.

48. Putting It All Together

49. Using Summative Assessment Results in the Instructional Cycle Data review and discovery What is the current state of performance? Improving Teaching and Learning Possible actions moving forward What should/will change? Recalling program, practice, policies, etc. What happened? Possible connections between performance and program, practice, policies, etc. How or why did the performance occur?

50. Research, Recall, Reflect, Respond Data review and discovery Possible actions moving forward Possible connections Program, practice, policies