The Smarter Balanced Assessment Consortium

1. TheSmarter Balanced Assessment Consortium Stuff You Should Know: Introductory Information for Parents, Policy Makers, Educators and Community Members Michael Hock Director of Educational Assessment Vermont Agency of Education

2. Each state pays for its own assessments • Each state bears the burden of test development; no economies of scale Why do we need new tests? Based on state standards • Students in many states leave high school unprepared for college or career; Limited comparability of results across states Heavy use of multiple choice • Inadequate measures of complex skills and deep understanding. Results delivered long after tests are given • Tests cannot be used to inform instruction or affect program decisions Accommodations for special education and ELL students vary • Difficult to interpret meaning of scores; concerns about access and fairness; Most administered on paper • Costly, time consuming, and challenging to maintain security

3. Smarter Balanced NECAP Content Standards How will SBAC be different from NECAP? Achievement Descriptors Test Format Item Delivery Testing Window Assessment Types Results Turnaround Item Types

4. Who is Smarter Balanced? • 25 member states and territories • 22 Governing States, 1Advisory State, 1 Affiliate Member • Washington state is fiscal agent • WestEd provides project management services

5. What’s the new assessment consortium all about? Is it really smarter and more balanced? The purpose of the Consortium is to… • Develop a comprehensive and innovative assessment system for grades 3-8 and high school in English language arts and mathematics aligned to the Common Core State Standards, so that... • ...students leave high schoolprepared for postsecondary success in college or a career through increased student learning and improved teaching • The assessments shall be operationalacross Consortium states in the 2014-15 school year

6. What’s so “balanced” about Smarter Balanced? Summative assessments Benchmarked to CCSS; Combine Computer Adaptive and Performance Tasks Teachers and schools have information and tools they need to improve teaching and learning Common Core State Standards specify K-12 expectations for college and career readiness All students leave high school college and career ready Interim assessments Flexible, open, used for actionable feedback Teacher resources for formative assessment practices to improve instruction

7. What would a "smarter balanced" school year look like? English Language Arts and Mathematics, Grades 3 – 8 and High School BEGINNING OF YEAR END OF YEAR Last 12 weeks of year* DIGITAL CLEARINGHOUSE of formative tools, processes and exemplars; released items and tasks; model curriculum units; educator training; professional development tools and resources; scorer training modules; and teacher collaboration tools. INTERIM ASSESSMENT INTERIM ASSESSMENT Computer Adaptive Assessment and Performance Tasks Computer Adaptive Assessment and Performance Tasks • PERFORMANCE • TASKS • Reading • Writing • Math END OF YEAR ADAPTIVE ASSESSMENT Scope, sequence, number, and timing of interim assessments locally determined Optional Interim assessment system— Summative assessment for accountability * Time windows may be adjusted based on results from the research agenda and final implementation decisions.

8. What’s the difference between summative, interim and formative assessments? • Summative Assessments are administered at the end of a specific unit or period of learning, generally near the end of a school year. They are designed to “sum up” how much the student has learned over that period of time, and to determine if the student’s achievement is sufficient to meet standards or pre-defined learning expectations. • Interim Assessments are similar to summative assessments in terms of content but are designed to be administered more frequently in order to determine if students are on track to meet end of grade/unit standards, or to provide additional support or mid-course correction if needed. • Formative Assessments are embedded in the day to day, minute to minute interactions between teachers and students. They are used to gauge the pacing of instruction and to determine if students are ready to move on to new material or if they need additional instruction. Formative assessments are often based on strategic questioning strategies, probes, short quizzes or performance events.

9. What is computer adaptive testing? The computer selects the first question for the student (Grade Level; Moderate Difficulty) The student answers correctly The student answers incorrectly The Computer Adaptive Assessment Algorithm Vastly Simplified The computer selects a harder question The computer selects an easier question Process is repeated until the computer arrives at the best possible estimate of the student’s achievement

10. Increased precision • Provides accurate measurements of student growth over time What's so special about Computer Adaptive Testing (aka CAT)? Tailored for Each Student • Item difficulty based on student responses Increased Security • Larger item banks mean that not all students receive the same questions Shorter Test Length • Fewer questions compared to fixed form tests Faster Results • Turnaround time is significantly reduced Mature Technology • GMAT, GRE, COMPASS (ACT), Measures of Academic Progress (MAP)

11. Summative Assessment (Computer Adaptive) What is a summative assessment? • Assesses the full range of Common Core in English language arts and mathematics for students in grades 3–8 and 11 (interim assessments can be used in grades 9 and 10) • Measurescurrent student achievement and growth across time, showing progress toward college and career readiness • Can be given once or twice a year (mandatory testing window within the last 12 weeks of the instructional year) • Includes a variety of question types: selected response, short constructed response, extended constructed response, technology enhanced, and performance tasks

12. Performance Tasks What role will performance tasks play? • Extended projects demonstrate real-world writing and analytical skills • May include online research, group projects, presentations • Require 1-2 class periods to complete • Included in both interim and summative assessments • Applicable in all grades being assessed • Evaluated by teachers using consistent scoring rubrics “ The use of performance measures has been found to increase the intellectual challenge in classrooms and to support higher-quality teaching. - Linda Darling-Hammond and Frank Adamson, Stanford University ”

13. How long are the summative tests? About the same total time as NECAP, but keep in mind that the new tests will measure more than the current assessments

14. Interim Assessment (Computer Adaptive) What is an interim assessment? • Optional comprehensive and content-cluster assessment to help identify specific needs of each student • Can be administered throughout the year • Provides clear examples of expected performance on Common Core standards • Includes a variety of question types:selected response, short constructed response, extended constructed response, technology enhanced, and performance tasks • Aligned to and reported on the same scale as the summative assessments • Fully accessible for instruction and professional development

15. Digital Professional Development Library What are formative assessment tools and professional resources? • Research-based, on-demand tools and resources for teachers • Aligned to Common Core, focused on increasing student learning and enabling differentiation of instruction • Professional developmentmaterials include model units of instruction and publicly released assessment items, formative strategies • Developed by teams of educators from the member states “ ” Few initiatives are backed by evidence that they raise achievement. Formative assessment is one of the few approaches proven to make a difference. - Stephanie Hirsh, Learning Forward

16. Online Reporting How will results be reported? “ • Static and dynamic reports, secure and public views • Individual states retain jurisdiction over access and appearance of online reports • Dashboard gives parents, students, practitioners, and policymakers access to assessment information • Graphical display of learning progression status (interim assessment) • Feedback and evaluation mechanism provides surveys, open feedback, and vetting of materials ” Data are only useful if people are able to access, understand and use them… For information to be useful, it must be timely, readily available, and easy to understand. - Data Quality Campaign

17. “Students can demonstrate progress toward college and career readiness in English Language arts and literacy.” What are the English Language Arts reporting areas (aka “claims”)? • “Students can demonstrate college and career readiness in English language arts and literacy.” Overall Claim for Grades 3-8 • “Students can read closely and analytically to comprehend a range of increasingly complex literary and informational texts.” Overall Claim for Grade 11 • “Students can produce effective and well-grounded writing for a range of purposes and audiences.” Claim #1 - Reading • “Students can employ effective speaking and listening skills for a range of purposes and audiences.” Claim #2 - Writing • “Students can engage in research and inquiry to investigate topics, and to analyze, integrate, and present information.” Claim #3 - Speaking and Listening Claim #4 - Research/Inquiry

18. “Students can demonstrate progress toward college and career readiness in mathematics.” What are the mathematics reporting areas (aka “claims”)? • “Students can demonstrate college and career readiness in mathematics.” Overall Claim for Grades 3-8 • “Students can explain and apply mathematical concepts and interpret and carry out mathematical procedures with precision and fluency.” Overall Claim for Grade 11 • “Students can solve a range of complex well-posed problems in pure and applied mathematics, making productive use of knowledge and problem solving strategies.” Claim #1 - Concepts & Procedures • “Students can clearly and precisely construct viable arguments to support their own reasoning and to critique the reasoning of others.” Claim #2 - Problem Solving • “Students can analyze complex, real-world scenarios and can construct and use mathematical models to interpret and solve problems.” Claim #3 - Communicating Reasoning Claim #4 - Modeling and Data Analysis

19. Access by Design What supports will be available for special populations? “ • Accurate measures of progress for students with disabilities and English Language Learners • Accessibility and Accommodations Work Group engaged throughout development • Outreach and collaboration with relevant associations • Universal Design • Embedded Digital Accessibility Features Common- Core Tests to Have Built-in Accommodations - June 8, 2011 ”

20. How will digital technologies improve assessment for special populations? Computer-delivered assessments provide an array of new opportunities to improve the assessment experience for students with special assessment needs, including students with disabilities and ELLs: • An expanded notion of Universal Design • Emphasis on embedded digital accessibility tools that decrease the need for locally provided accommodations • Integration of assessment delivery system with assistive technologies • Vastly improved student engagement

21. A Conceptual Model for SBAC Accessibility Available to ALL Students Available with IEP or 504 Plan

22. A Conceptual Model for SBAC Accessibility

23. A Conceptual Model for SBAC Accessibility (Math),

24. A Conceptual Model for SBAC Accessibility Embedded American Sign Language, Braille, Closed Captioning, Text to Speech (Reading) Non-Embedded Abacus, Alternate Response Options, Calculator, Multiplication Table, Print on Demand, Read Aloud, Scribe, Speech--to -Text)

25. Does CAT have any special advantages for special students? • PRECISE – Unlike fixed form tests, CAT is precise at all ability levels, not just at the proficient cut score • EFFICIENT– Takes less items than fixed form tests to reach a valid and reliable estimate of ability, reducing testing time by as much as 50%; results are available almost immediately. • RESPONSIVE & HUMANE – CAT algorithm produces an individual item set for each student, adapting after each student response; item set comprised of items the student CAN and CAN’T answer in equal proportion • Secure and Adaptable – CAT eliminates most test security concerns of fixed form tests allowing for longer test windows and small group or individualized administrations

26. What do school boards need to know? • COST – All the direct costs for development, administration, scoring and reporting the Smarter Balanced Assessments will be covered by the State of Vermont • SCHOOL/DISTRICT RESPONSIBILITIES– (1) computers, (2) access to the internet, (3) headphones or earbuds, (4) tech support for set-up, (5) staff for test administration, (6) released time for training test administrators • FIELD TESTING – To ensure that the new technologies developed for Smarter Balanced work as intended, the consortium is currently field-testing the assessment with more than 3 million students in 22,000 schools (nearly 5 thousand students in 28 Vermont schools). • TECHNOLOGICAL READINESS – According to current estimates, all but two or three Vermont schools have enough computers, sufficient internet bandwidth and minimum tech support to be ready for the first official administration of the test in Spring 2015 • TECHNOLOGICAL DEMANDS - SBAC will place mild to moderate demands on the technology infrastructure of typical US schools. • Geoff Fletcher, SETDA: “If a school can’t handle these tests then the school has much bigger problems because their students will not be able to access all the excellent digital learning tools that are being introduced every day.”

27. Sample Items? Can we see some of those? • Showcases the variety of item types: • Selected response • Constructed response • Technology enhanced • Performance tasks http://sampleitems.smarterbalanced.org/itempreview/sbac/ELA.htm

28. Practice Tests? Where are they located? http://sbac.portal.airast.org/practice-test/

29. Where can I learn more about Smarter Balanced? • On the web @ www.SmarterBalanced.org • Sign up for the Smarter Balanced e-newsletter • Follow the consortium on Twitter @SmarterBalanced • On the VT AOE web @ http://education.vermont.gov/new/html/sbac.html • E-Mail Michael @ Michael.Hock@state.vt.us

30. Questions What else would you like to know?