Smarter Balanced Assessment System

1. Smarter Balanced Assessment System Dr. Cristi Alberino,ELA Assessment Consultant, CSDE Abe Krisst, CMT Education Consultant, CSDE Education Institutes of Higher Education Symposium April 27, 2012

2. The Assessment Challenge ...to here? Common Core State Standards specify K-12 expectations for college and career readiness All studentsleave high school college and career ready ...and what can an assessment system do to help? How do we get from here...

3. Each state procures its own assessment system • Each state bears the burden of test development; no economies of scale Summative Assessments Today Measure proficiency against state standards, not agreed-upon standards • Students often leave high school unprepared to succeed in entry-level college courses Usually heavy reliance on multiple choice questions • Poor measures of demonstration of skills and complex cognitive performance Results often delivered months after tests are given • Tests cannot be used to inform instruction or affect program decisions Accommodations for special education and ELL students vary • Difficult to interpret meaning of scores; concerns about access and fairness Most administered on paper • Costly, time consuming, and challenging to maintain security

4. Next Generation Assessments Source: Federal Register / Vol. 75, No. 68 / Friday, April 9, 2010 pp. 18171-85 • More rigorous tests measuring student progress toward “college and career readiness” • Have common, comparable scores across member states • Provide achievement and growth information to help make better educational decisions and professional development opportunities • Assess all students, except those with “significant cognitive disabilities” • Administer online, with timely results • Use multiple measures

5. A National Consortium of States • 27 states representing 43% of K-12 students • 21 governing, 6 advisory states • Washington state is fiscal agent

6. The Purpose of the Consortium To develop a comprehensive and innovative assessment system for grades 3-8 and high school in English language arts and mathematics aligned to the Common Core State Standards, so that... ...students leave high school prepared for postsecondary success in college or a career through increased student learning and improved teaching [The assessments shall be operational across Consortium states in the 2014-15 school year]

7. Remediation in Higher Education Nationally, 42% of students at 2 year institutions took a remedial course 70% of students who took 1 or more remedial reading classes did not get a college degree within 8 years 51% of students who took the ACT were not ready for college level reading

8. Comparability PARCC and Smarter Balanced working together to ensure that proficiency standards and data will be comparable and portable, A joint advisory committee has been formed to advise both consortia on this issue. Addressing Higher Education Concerns Relevance Stability Utility Developing a business plan for post-2014 with funding Governing states actively involved in determining future of Smarter Balanced Data to support tailored instruction for students not on track to college/career readiness. Higher education faculty involved in assessment design to ensure that the assessments are true to Common Core standards and higher education expectations. Quality Computer adaptive technology and performance tasks. Common protocols for item development: accessibility, language/cultural sensitivity, accommodations, etc.

9. Reaching the Goal: Expectations of Higher Education

10. Smarter Balanced Design

11. A Balanced Assessment System Summative assessments Benchmarked to college and career readiness Teachers and schools have information and tools they need to improve teaching and learning Common Core State Standards specify K-12 expectations for college and career readiness All students leave high school college and career ready Teacher resources for formative assessment practices to improve instruction Interim assessments Flexible, open, used for actionable feedback

12. Test Question Types and Design • Items and the collection of items MUST measure deeper understanding and application of conceptions. • Therefore a variety of item types are necessary: • selected response, short constructed response, extended constructed response, technology enhanced, and performance tasks

13. Changes to English Language Arts Assessment

14. ELA Reading Deficiencies • Inadequate understanding of words • Inadequate background knowledge • Lack of knowledge about writing conventions • Inability to recall information given verbally • Inability to understand implicit information

15. Education Policy Improvement Center (EPIC) • Survey of 1897 postsecondary instructors • both 2 and 4 year schools • variety of subject areas: ELA, Math, science, social science, business, computer technology, healthcare • CCSS English Language Arts and Literacy standards are applicable and important for postsecondary readiness

16. Are the ELA standards as a whole a coherent representation of the fields of knowledge necessary for success in your course?

17. Reading and writing Grounded in Evidence From Text • Students should be taking stances and using evidence from sources to support their positions (verbally and in writing) • Writing prompts should be tied to texts • In other words, the student must have read and analyzed a text to respond to the prompt.

18. Several Key Shifts • Building knowledge through content-rich nonfiction and informational texts • Reading and writing grounded in evidence from text • Regular practice with complex text and its academic vocabulary

19. ACT Reading Between the Lines The complexity of what students can read is the greatest predictor of success in college

20. Building Knowledge Through Content-Rich Nonfiction and Informational Texts Range of Text Types • Make sure that students are exposed to a wide range of text types including informational text (HS – 70%/30%). Text Complexity • Make sure students are exposed to grade level text complexity – gradient increase with support.

21. Preparing Teachers to Use CCSS Text Complexity is the key!

22. Regular practice with complex text and academic vocabulary • Tier 2 Vocabulary: Words that add to students’ language ability (e.g., maintain, fortunate, required, tend, contrast/compare, insisted, summarize) • Tier 3 Vocabulary: Words that are needed in a content area (e.g., isotope, peninsula, photosynthesis, cubism, isosceles triangle)

23. Preparing Teachers to Use CCSS • Do not “fragment standards” • Do not arrange standards into new categories • Use cluster headings and domain names • Writing: Production and Distribution of Writing • Research to Build Knowledge

24. Total English language arts / literacy (Grades 3-8) “Students can demonstrate progress toward college and career readiness in English language arts and literacy.” Total English language arts / literacy (High School) “Students can demonstrate college and career readiness in English language arts and literacy.” Assessment Claims for English Language Arts/Literacy (a/o 12/27/11)

25. Reading • “Students can read closely and analytically to comprehend a range of increasingly complex literary and informational texts.” Assessment Claims for English Language Arts/Literacy Writing (a/o 1/6/11) • “Students can produce effective and well-grounded writing for a range of purposes and audiences.” Speaking/Listening • “Students can employ effective speaking and listening skills for a range of purposes and audiences.” Research/Inquiry • “Students can engage in research and inquiry to investigate topics, and to analyze, integrate, and present information.”

26. Understanding the Process Overarching Claims 3-8 and High School Assessment Claims Targets Standards

27. Claim 1: Reading Informational TextTarget 12: Analysis Within or Across Texts Model is based on standards from Listening/Speaking, Writing, Reading Literature and Informational Text Evidence: • students conduct research • evaluate sources, analyze and integrate • write a short report with evidence cited • use precise language for purpose and audience • deliver oral presentation

28. How Does this Task Contribute to the Sufficient Evidence for the Claim? In order to complete the PT, students Make inferences and summarize using key details in text. Analyze information presented in multiple texts. Analyze information delivered orally and visually Conduct short research on a topic, analyze and interpret the information, and cite evidence about how it supports a concept Organize, compose, and deliver oral presentations using precise language appropriate to purpose and audience. Use visual or audio information to enhance oral presentations

30. Mathematics

31. Total mathematics (Grades 3-8) “Students can demonstrate progress toward college and career readiness in mathematics.” Total mathematics (High School) “Students can demonstrate college and career readiness in mathematics.” (not yet released)

32. Concepts and Procedures • “Students can explain and apply mathematical concepts and carry out mathematical procedures with precision and fluency.” Assessment Claims for Mathematics Problem Solving (a/o Round 1 – released 8/29/11) • “Students can frame and solve a range of complex problems in pure and applied mathematics.” Communicating Reasoning • “Students can clearly and precisely construct viable arguments to support their own reasoning and to critique the reasoning of others.” Data Analysis and Modeling • “Students can analyze complex, real-world scenarios and can use mathematical models to interpret and solve problems.”

33. Shifts Evident in CCSSM • FOCUS • FOCUS • FOCUS • Coherence • Fluency • Deep Understanding • Application • Intensity

34. Key Fluencies

35. Mile-wide/inch-deep traditions Attaining CCSSM balance conceptual understanding procedural skill application Connecting content and practices Grain size in CCSSM Psychometric culture Implementation Challenges

36. Changes to Mathematics Assessment

37. Assessment • What will happen to CMT and CAPT? • CT has submitted an application for an NCLB waiver. • School year 2014/2015, SMARTER Balanced Assessment Consortium (SBAC) assessment system operational for students in Grades 3-8 and 11.

38. Item Exemplars: Performance Task Performance Task drawn from the Ohio Performance Assessment Project.

39. Item Exemplars: Performance Task (cont’d) Performance Task drawn from the Ohio Performance Assessment Project.

40. Item Exemplars: Performance Task (cont’d) Performance Task drawn from the Ohio Performance Assessment Project.

41. Essential Properties of Tasks That Assess Claim 1 Conceptual Understanding and Procedural Fluency Tasks assessing this category will include short items, multiple-choice, other selected-response, and short constructed-response items, that focus on a particular skill or concept. They will also include items that require students to translate between representations of concepts (words, diagrams, symbols) and items that require the identification of structure. Aspects of Claim 1 can also be examined in the context of student work on more complex tasks.

42. Sample Item

43. Sample Item

44. Sample Item

45. Sample Item Sarah is 12 years old. • George is g years old. • Sarah is 3 times as old as George. For numbers 1a – 1c, choose Yes or No to indicate whether each statement is true. 1a. George’s age, in years, can be represented by the expression 12 ÷ 3. 1b. George is 15 years old. 1c. Sarah’s age, in years, can be represented by the equation 12= 3 × g.

46. OECD, 2009. Sample questions from OECD’s PISA Assessments. ISBN 978-92-64-05080-8.

47. Essential Properties of Tasks That Assess Claim 2Problem Solving Evidence for Claim 2 depends on tasks that present non-routine problems where a substantial part of the challenge is in deciding what to do, and which mathematical tools to use; involve chains of autonomous reasoning, taking a successful student at least 5 to 10 minutes (depending on the age of the student and complexity of the task), including explanation of assumptions and conclusions as well as the use of representational and procedural skills.

48. Essential Properties of Tasks That Assess Claim 3 Communicating Reasoning Evidence for Claim 3 depends on tasks that present a situation in which either propositions are given or students are encouraged to make their own conjectures; ask students to test propositions or conjectures with specific examples; ask students to construct, autonomously, chains of reasoning that will justify or refute the propositions or conjectures; these chains should typically take a successful student 10 minutes or more. (Times will be somewhat shorter for younger students, but still giving them time to think and explain.)

49. Gavin, M. K., Casa, T. M., Chapin, S., Copley, J. V., & Sheffield, L. J. (2008). Project M2: Using Everyday Measures: Measuring with the Meerkats from Project M2: Mentoring Young Mathematicians series.

50. Sample Item