Supporting Changes in Teacher Practice with a Learning Progression: Results of the Elevate Study

1. Supporting Changes in Teacher Practice with a Learning Progression: Results of the Elevate Study Erin Marie Furtak University of Colorado at Boulder June 30, 2017 University of South Bohemia Cesky Budejovice

2. 2000-2002

3. Lingering Questions ? Furtak (2012)

4. Professional Development Teacher Cognition Activities & Assessments Classroom Practices Student Achievement

5. Learning Progressions Hierarchical representations of how student ideas develop in a content domain Correct, scientifically accurate ideas at top Everyday ideas a.k.a. misconceptions at the bottom Usually used for curriculum and assessment development

6. Formative Assessment Professional Development Teacher Cognition Teacher Cognition Activities & Assessments Classroom Practices Classroom Practices Student Achievement Student Achievement Learning Progression

8. Formative Assessment Tools Practices Participants

9. Formative Assessment Tool Open-ended questions about natural selection investigation (model of Darwin’s Finches) Practices Participants

11. Formative Assessment Tool Open-ended questions about natural selection investigation (model of Darwin’s Finches) Practices How can we capture these? Participants High School Biology Teachers

12. Teacher 31 Teacher 34 Teacher 33

13. Formative Assessment Tool Open-ended questions about natural selection investigation (model of Darwin’s Finches) Practices How can we capture these? Participants High School Biology Teachers

14. Leave blank for now As you watch the videos, write down a few notes and observations about what you notice about each teacher’s classroom practices.

15. Beans and Beaks Formative Assessment Question 5 If the environmental conditions select for the elimination (extinction) of a certain beak type, and the environment changes back to its original conditions, would the eliminated (extinct) beak type make a return?

16. Teacher 31

17. T31: So if the environment conditions select for the elimination of a certain beak type, like we keep talking about the wooden sampler spoon going away. And then the environment changes back to its original conditions, what would happen to the birds? What would happen to that bird population? [No student response] T31: So everybody pretty much thought while some birds might come back that generally speaking the ones that went extinct would not return. Now why wouldn’t they return? Why wouldn’t they just come back? Student: Their genes are gone.

18. T31: Huh? Their genes are gone? You mean to tell me if those birds disappeared I couldn’t take some water, dump it on some bird fossils and they wouldn’t suddenly pop out of the ground like little plants? Student: That would not happen.

19. T31: That would not happen. No. No. Once a bird population has disappeared, or any population, they wouldn’t come back. The wooly mammoth existed during the ice age 10,000 years ago. Whether it was because of the change in environmental conditions, or there are theories that humans hunted them out of existence, whatever the case may be, they’re gone. If we all of a sudden had an ice age would they re-appear? Student: No. T31: No. Those genes have left the population. What would be the only possible way that those genes could possibly return if conditions returned to favor what they were doing?

20. Teacher 34

21. T34: If the environmental conditions select for the elimination of a certain beak type and the environment changes back to its original conditions would the eliminated or extinct beak type make a return? Give me thumbs up if you think yes. <pause> Give me thumbs down if you think no. <pause> A lot of us said yes. If you said yes can you explain why? Lisa: Because that’s how that beak came to be in the first place. To get the food, each finch after several centuries grows a beak to help them get the most food possible.

22. T34: Hold on. Do they grow that beak to help them get, they’re like I want this kind of beak so they grow that to help them get it? What do you mean by that? Lisa: Genes kind of change. Addie: Evolve. Lisa: Yeah, evolve after several generations. T34: Can anyone help Lisa? Yeah, Mason.

23. Mason: I said maybe. Because the beak would evolve back again to that beak size. T34: Ok. Here’s the answer. The big, big glaring answer to number 5 is no. So here’s the deal. Thank you for being a brave underdog and being like “uh I thought no.” Here’s the deal. Could that beak type return? Well there’s one way you could say yes but it’s not very likely. So why is the answer no could someone explain it to me? Zac, yeah?

24. Teacher 33

25. T33: If the environmental conditions select for the elimination of a certain beak type and then we go back to our original world of seeds, what would happen to the birds? First of all, what happens if you eliminate a beak type? What word do we use for that? Casey: Extinct.

26. T33: They go extinct. So if we lose these guys right here. If they’re gone they totally died out. They’re gone. Then we get all those seeds back. Will the birds change their beaks os they can be the same gene type? A mutation in the gene would allow that beak type to come back? While some birds would be more successful in the changed environment the extinct ones would come back? What do you guys think? Nora: It would come back or wouldn’t come back?

27. T33: This says they would. Do you guys think that C is possible? David: Maybe. T33: If the wooden samplers are gone can they come back? Ross: No Maddie: There’s a chance that a genetic variation mutation could happen.

28. T33: Ok so Maddie is saying maybe there’s a chance for a genetic mutation to kind of happen where you might get that beak type back. It’s a little complicated I think in that respect. Overall, if we lose species overall on the planet, they’re gone. Do we get them back? Cora: No. T33: No. Unlikely although there’s some pretty crazy genetic stuff that might happen. You usually know. we can have some other mutations of other beak types. This guy is really not coming back.

29. What differences do you observe between the formative assessment practices these three teachers used with their students? Discuss with a partner for 5 minutes Share comments/thoughts 31 34 33 Teacher 31 Teacher 34 Teacher 33

30. What do YOU think... Go to Menti.comand enter in the code 57 74 38

31. Hold on to those ideas...we’ll get back to them later in the slide show.

32. Lingering Questions ? Furtak (2012)

33. Reflect on Enactment Formative Assessment Design Cycle Enact Tools Explore Student Ideas Practice Using Tools Develop Tools Furtak, 2012; Furtak, Morrison & Kroog, 2014; Furtak & Heredia, 2014

34. Reflect on Enactment Formative Assessment Design Cycle Teachers learn about ‘big ideas’ in science and how student understanding of these ideas develops Enact Tools Explore Student Ideas Practice Using Tools Develop Tools

35. Reflect on Enactment Formative Assessment Design Cycle Enact Tools Explore Student Ideas Practice Using Tools Develop Tools Teachers develop formative assessments designed to elicit student thinking related to the ‘big ideas’

36. Reflect on Enactment Formative Assessment Design Cycle Enact Tools Explore Student Ideas Practice Using Tools Develop Tools Teachers rehearse enactment of the activities and anticipate how they will respond to certain student ideas

37. Reflect on Enactment Formative Assessment Design Cycle Enact Tools Explore Student Ideas Teachers enact the activities with their own students, the enactment is videotaped, and copies of student work are collected Practice Using Tools Develop Tools

38. Reflect on Enactment Formative Assessment Design Cycle Teachers reflect on evidence of enactment to determine effectiveness and next steps Enact Tools Explore Student Ideas Practice Using Tools Develop Tools

39. Reflect on Enactment Formative Assessment Design Cycle Sequencing of ‘correct’ ideas Enact Tools Explore Student Ideas Practice Using Tools Develop Tools Increasingly sophisticated ideas

40. Educative Learning Progressions as Tools for Teacher Development 2010-2014

41. Operationalizing Formative Assessment

42. More Aligned with Formative Assessment More Traditional/ Didactic Where would you place these teachers for each category? Why do you think so? Designing Tasks Eliciting Student Ideas Interpreting Student Ideas Providing Feedback

43. What do YOU think... Go to Menti.comand enter in the code 57 74 38

44. Reflect on Enactment Tools Better Identifying and Interpreting of student ideas Participants Enact Tools Explore Student Ideas Better Formative Assessment Task Design Practice Using Tools Develop Tools Increase in student achievement Asking higher-quality questions Practices Giving Higher-quality feedback

45. Research Questions To what extent does participation in the Formative Assessment Design Cycle support increases in the quality of teachers’ formative assessment task design, questions to elicit student thinking, interpretation of student ideas, and feedback? To what extent does teachers’ proficiency in these formative assessment abilities predict changes in student achievement?

46. Study Design

47. Reflect on Enactment Participants & Intervention Enact Tools Explore Student Ideas • 9 biology teachers at 3 high schools • Students NBaseline= 472; NYear 3= 428 • Monthly, 60-90 minute professional development sessions • Separate meetings at • each partner school Practice Using Tools Develop Tools

49. Designing Formative Assessment Tasks Likert-Scale items, 1=Traditional, 5=High-quality formative assessment 1.What type of space is there in the activity for students to share their ideas? 2.What is your impression of the type of instruction that would accompany this activity, based on how it is written? 3.What type of knowledge is being elicited/promoted/targeted? 4.What type of information about student ideas does this activity seem designed to provide? 5.What is the potential of this activity to make students’ scientific understandings explicit? 6.What is your impression of how difficult or easy it might be to interpret these scientific understandings? • Six raters (all biology teachers trained in formative assessment) applied ratings to all tasks collected • ICC=0.80 to 0.96.