OREGON’S SCIENCE ACHIEVEMENT LEVEL DESCRIPTORS AND STANDARDS

1. OREGON’S SCIENCE ACHIEVEMENT LEVEL DESCRIPTORS AND STANDARDS Policy Definitions, Achievement Level Descriptors, Science Achievement Standards (Cut Scores) for General Education and for Alternate Assessment

2. Recommending Adoption of ScienceAchievement Standards and Level Descriptors • Overview • Kathleen Vanderwall, OAIS • External Evaluation • Mary Seburn, Educational Policy Improvement Center • Implementing 2009 Content Standards for Instructional Improvement • Monument School District, Rachel Aazzerah • North Clackamas School District, Meagan Sternberg

4. Background • Policy Definitions are general statements that describe rigor across grade levels and content areas and provide an overarching definition for each achievement level.  They are used for all subjects. • Achievement Level Descriptors (ALDs) explain a student’s knowledge and skills at a specific Achievement Level by grade group and subject and are a key element in the establishment of cut scores. • ODE focused the Achievement Standards Verification panelists’ attention by using outside references with data on national and international benchmarks.

5. Achievement Level Descriptors • Specify the knowledge, skills and abilities a student needs to know in order to be classified as: • Does Not Yet Meet • Nearly Meets • Meets • Exceeds

6. Achievement Standards (Cut Scores) • Scores on Oregon’s scale. • Describe what students should be able to do in each achievement level. • Participants actively discuss their expectations of the “target student” in each achievement level using a booklet of 70 Oregon test items ordered from easiest to most difficult.

7. Communication with the Field • May-September 2010 Moving Science Forward Professional Development • 2009, 2010 October Statewide In-Service- new standards and assessments presentations • September 2009-present Science Monthly Updates, AA Updates, Superintendents Pipeline • February 2011 Draft 2009 Test Specifications and Blueprint Documents posted to web • January-May 2011 Announcements regarding reporting, assessments, scoring guides and Science Standard Verification • August-September 2011 Survey • October 2011 – May 2012 Communication Strategy

8. Framework • General Policy Definitions (adopted March 2010) • Science Policy Definitions (provide consistency across grades-approved by Science Panel January 2011) • Science Achievement Level Descriptors for grade 5, 8, HS (adopted May 2011) • Final Achievement Level Descriptors and Achievement Standards linked to outside references will be available to inform partners.

9. External Evaluation Dr. Mary Seburn Director Research, Design and Analytics Division Educational Policy Improvement Center

10. Standards Verification Process • Modified Bookmarking method • Comparative data from NAEP, PISA, OUS, other states • Nearly Meets, Meets, and Exceeds • Grades 5, 8, and high school Science • 24 stakeholders • Three days • Recommendedincreasedachievement standards for all grade levels • Except for “Nearly Meets”; current standards confirmed

11. Current and Recommended Achievement Standards Note: No changes were recommended to the Nearly Meets Achievement Standard.

12. External Evaluation The Educational Policy Improvement Center (EPIC) • Documented evidence of technical adequacy • Procedural validity, as observed • Standards for Educational and Psychological Testing • NCLB Peer Review • Best practice • Materials review • Observations • Interviews • Evaluations

13. Evaluation Criteria Twenty-one evaluation elements total, all met: • Panelist Selection • Panelist and Facilitator Training • Methodology • Process • Documentation

14. Evaluation Outcomes:Participant feedback on the training was positive “The training materials were helpful.” “I am confident I understand my role in the standards verification process.” “The training clearly identified the goals for the standards verification procedure.”

15. Evaluation Outcomes:Participant feedback on the process was positive “I understood how to place my bookmarks.” “Overall, I am satisfied with my group's final bookmarks.” “I am confident that the bookmark procedure used produced valid Achievement Standards.” “The bookmark procedure was well described.”

16. Evaluation Outcomes:Open-ended participant feedback was positive • “Steps in the process were very good, group and individual work time allowed for flexibility and allowed us to come to consensus.” • “I would love to do this again. I can’t think of anything they could do better. Very successful.” • “I really believe in the process, it is difficult, but it’s why democracy works.”

17. Implementing 2009 Content Standards for Instructional Improvement Rachel Aazzerah 7-12 Science/Math Teacher Professional Learning Community Leader/Data Driven Decision Making Coordinator Monument School District

18. Middle School Water Testing Explain how individual organisms and populations in an ecosystem interact and how changes in populations are related to organisms.   Interaction and Change Life Science (6.2L.2)

19. Middle School Water Testing Evaluate natural processes and human activities that affect global environmental change and suggest and evaluate possible solutions to problems.  Interaction and Change Earth and Space Science (7.2E.3)

20. High School Forensics Explain and apply laws of heredity and their relationship to the structure and function of DNA.  Structure and Function Life Science (H.1L.3)

21. Implementing 2009 Content Standards for Instructional Improvement Meagan SternbergK-12 Science Instructional Coach North Clackamas School District

22. Engineering Design • North Clackamas School District is in the middle of reviewing and realigning science courses with a focus on making sure students are exposed to Engineering Design content, practices and work samples at each grade level. • This is challenging in some science courses. • However, we feel it is critical work with the new focus on Engineering and Technology at the national level.

23. Engineering Design • Happy Valley and Rock Creek Middle Schools have an elective called “Engineer It!”. • At HVMS, almost all students take this class.

24. Science Achievement Standards • Raising the Science Achievement Standards increases the rigor and importance of teaching and learning science in Oregon schools.

25. Next Steps • State Board adoption of recommended achievement standards and achievement level descriptors • Peer Review Submission- December 2011

26. Thank you! Questions?

27. Alternate Achievement Standards for Science Dianna Carrizales-Engelmann Brad Lenhardt Office of Student Learning and Partnerships Daniel Farley University of Oregon

29. Alternate Assessment

30. Comparing the General Assessment to the Alternate Assessment

31. Science 2011

32. Content Prompt

33. Student Materials

34. Alternate Assessment Achievement Standard Setting Process • University of Oregon Behavioral Research and Teaching Dr. Gerald Tindal • Bookmarking process • 13 educators representing 9 counties • SPED and content expertise • 3 rounds • Review of impact data • Recommendations: Both increases and decreases related to refinements to the new content standards

35. Science Alternate Assessment Proposed Cutscores and Directional Change

36. Science Alternate Assessment Proposed Cutscores and Impact Data

37. Alternate Science Policy Definitions

38. Recommendation Justification • Rigor of process • Appropriateness of panelists • Quality of personnel/oversight • Thoroughness of training • Validity of assessment • Panelist Confidence Assessment data

39. Thank you! Questions?

40. General Policy Definitions • Does Not yet Meet: Students do not demonstrate mastery of grade-level knowledge and skills required for proficiency. • Nearly Meets: Students demonstrate partial mastery of grade-level knowledge and skills required for proficiency. • Meets: Students demonstrate mastery of grade-level knowledge and skills required for proficiency. • Exceeds: Students demonstrate mastery of grade-level knowledge and skills exceeding the requirement for proficiency.

41. Proposed Science Policy Definitions • Does Not Yet Meet: Students demonstrate limited mastery of science, scientific inquiry and engineering design knowledge and skills, through the application of basic concepts with occasional success. • Nearly Meets: Students demonstrate partial mastery of science, scientific inquiry and engineering design knowledge and skills, through the application of basic concepts with regular success. • Meets: Students demonstrate mastery of science, scientific inquiry and engineering design knowledge and skills, through the interpretation and application of grade-level concepts with consistent success. • Exceeds: Students demonstrate mastery of science, scientific inquiry and engineering design knowledge and skills, through the interpretation and application of advanced concepts with consistent success.

42. Definitions • Proficient: A student who is proficient has solid academic performance. • Mastery: A student who has mastered a subject is able to clearly explain aspects of the subject to fellow students.