Using Data to Improve Instruction in Science:

1. Using Data to Improve Instruction in Science: Do we teach in light of the data or in spite of the data? Michael Horton Coordinator Riverside County Office of Education

2. Three Questions • Do we teach in light of the data? • Do we teach in spite of the data? • Do we teach in an absence of data?

3. The Answer to these questions is . . . Yes!

4. In Light of the Data The idea that is sweeping the country: Professional Learning Communities is the ultimate example of teaching in light of data.

5. Professional Learning Communities PLCs are built upon 3 key questions: • What do we want students to learn? • How will we know if they learned it? • What will we do if they don’t?

6. Professional Learning Communities Which researchers support PLC? Rick DuFour Jon Saphier Rebecca DuFour Mike Schmoker Roland Barth Dennis Sparks Robert Eaker Rick Stiggins Michael Fullan Robert Marzano Larry Lezotte Doug Reeves Milbrae McLaughlin Tom Many “The most promising strategy for sustained, substantive school improvement is building the capacity of school personnel to function as a professional learning community.” - Milbrae McLaughlin

7. Professional Learning Communities Which groups support PLC? National Science Teachers Association (NSTA) Mid-Continent Research for Education and Learning (McREL) National Staff Development Council Association for Supervision and Curriculum Development (ASCD) Southwest Educational Development Laboratory National School Boards Association “Professional development programs should be based on the needs of science educators—of both individuals and members of collaborative groups.” -NSTA Position Statement

8. Real-World Example “X” High School in Riverside County- 2 Chemistry teachers decided to look at data and measure efforts to improve that data Began a program to complete 50+ chemistry labs at home beginning in 2002-2003 In 2002, had an API of 575, a statewide rank of 3, and identified by CDE as an “underperforming school” 66% low SES and rising, 40% of parents never graduated high school, high EL population

9. Real-World Example: The Data Chemistry Mean Scale Score in California 2006, 326 at PHS, 331 Geometry Mean Scale Score in California 2006, 312 at PHS, 281

10. Real-World Example: The Data

11. Real-World Example: The Data

12. In Spite of the Data • Real District “X” was scoring very well on the 8th grade science test. • 9th graders were required to take Integrated Science and test results were poor. • 10th graders took Biology and test results were excellent again. • This continued for at least 4 years. • Many teachers concluded that Integrated Science was the problem.

13. In Spite of the Data 2005-2006

14. In Spite of the Data • The good news is that in 2006-2007, they began to look at their curriculum and did 2 things: • Aligned the Integrated courses more closely with the Integrated 1 blueprint. • Tested to see if Earth Science would be a better option for 9th graders. • The 9th grade data improved.

15. In Light of the Data 2006-2007

16. More In Spite of the Data Based upon recent TIMMS data, “U.S. twelfth graders scored below the international average and among the lowest of the 21 TIMSS nations in both mathematics and science general knowledge in the final year of secondary school.” NAEP data show that California students score 11 points below national average on the 8th grade science test. On the 2005 NAEP, California 8th graders scored second-to-last in the country. California is below average nationally and the nation is below average worldwide in science. California is among a small minority of states that only require 2 years of science to graduate.

17. More In Spite of the Data 2 years 3 years 4 years

18. More In Spite of the Data Didactic Lecture: Cognitive science says that lecture and recall doesn’t work. “Research tells us that the human brain can hold about seven items in its short term working memory . . . The number of new items that students are expected to remember in the typical science lecture is vastly greater.” Wieman, 2007 Research in the classroom says that lecture and recall doesn’t work Interactive engagement versus performance, Hake 2002

19. More In Spite of the Data • Yet, we still teach primarily via didactic lecture and recall. • Recently, 1,500 classroom observations were logged and characteristics measured. This is what was found: • Classrooms in which research-based strategies were being used: 0.2% • Classrooms in which there was evidence of higher-order thinking: 3% • Classrooms in which fewer than half of students were paying attention: 85% • Classrooms in which students were completing worksheets: 52% • Classrooms in which non-instructional activities were occurring: 35% • In my duty of performing Williams Visits this year, I visited 500 classrooms at a variety of grade levels and a variety of subject areas. • - Number of classrooms in which I saw anything other than lecture, bookwork, and worksheets: 4

20. In the Absence of Data

21. In the Absence of Data

22. In the Absence of Data About 10 years ago, a reform movement began to switch physics to 9th grade. There was no data supporting this idea, only anecdote. Many schools and districts switched and did not collect any data to judge its effectiveness. To this date, no formal study has been done to measure its effectiveness. In 2001, Leon Lederman estimated that over 100 schools (60% private) had switched to physics first nationally. As of today, no data has been published showing its effect. A website now lists 299 schools teaching physics first, certainly not a complete listing. “Physics first schools are not quantitatively documenting the degree of their success” (Pasero, 2001).

23. In the Absence of Data • I have since collected data from California schools showing: • Physics first destroys physics test scores • Physics first hurts chemistry test scores • Physics first does not affect biology test scores • Physics first does not increase enrollment in higher physics classes • Analyzing the standards has also showed: • Physics standards are highest on Bloom’s Taxonomy • Very little chemistry and physics are needed to be proficient in the CA biology standards • All of the chemistry and physics needed is in the middle school standards • Calculators are not even allowed on the physics test, this is not a mathematics issue • Still, over 13,000 9th graders took the physics CST last year.

24. Good News • 13,000 is down from 16,000 in 2006; 17,000 in 2005; and 15,000 in 2004 • There were physics first schools that had 0% proficiency on the test yet no physics last schools with 0% proficiency on either 9th grade biology nor 11th grade physics. This is an improvement for those schools.

25. Conflicting Data: Homework “A typical homework-completing high school student will outperform students who do not do homework by 69% on standardized tests.” - Harris Cooper “Homework-completing junior high students outperform homework non-completers by 35%.  There seems to be no difference in scores in the elementary grades.” -Debbie Reese “[R]esearch fails to demonstrate homework's effectiveness as an instructional tool and recommended changing the “default state” from an expectation that homework will be assigned to an expectation that homework will not be assigned.” -Kohn, 2006 “According to Kohn, teachers should only assign homework when they can justify that the assignments are “beneficial” ideally involving students in activities appropriate for the home, such as performing an experiment in the kitchen. . .”

26. How to Use Data I said I taught him. I didn’t say he learned it. I taught my dog to whistle. I don’t hear anything. • Focus on student achievement • CST data is valuable, but too infrequent to change daily practice • Collaborate and share best practices • Set SMART goals • Measure progress • Review progress and make changes as necessary

27. How to Use Data • Questions to consider when looking at CST or other data: • (Adapted from a Dennis Fox presentation, Los Angeles County Office of Education) • How has the distribution of scores changed over the different subject areas over the last 3 years? • What is the “net gain” or “net loss” at each performance level for each subject area over the last 3 years? • What is the net gain or net loss for the combined performance levels (FBB/BB and Prof/Adv) for each subject area over the last 3 years? • Which subject area has made the most growth over the last 3 years? • Which subject area has made the least growth or no growth over the last 3 years?

28. How to Use Data • Questions to consider when looking at CST or other data: • (Adapted from a Dennis Fox presentation, Los Angeles County Office of Education) • During which year did the least growth occur for each subject? • What are possible contributing factors to the above answers? • How has the distribution of performance levels changed for different subgroups? • How does each change compare to your SMART goal? • What factor(s) will you change before your next common assessment?

29. Contact Information Michael Horton mhorton@rcoe.us Download PowerPoints and whitepapers and discuss issues, ask questions, or make comments about any of my presentations at: csta.notlong.com