Susan Kirch (PI) 1,3 , Catherine Milne (coPI) 1 , Anna Stetsenko (coPI) 2

1. Instructional Design Classroom-based analysis Teaching and Learning Concepts of Scientific Evidence: A Design-based Research and Development Study Susan Kirch (PI)1,3, Catherine Milne (coPI)1, Anna Stetsenko (coPI)2 Kara Naidoo1, Laura Paskell-Brown2 1New York University - Steinhardt School of Culture, Education and Human Development, 2The Graduate Center of the City University of New York, 3Correspondence to: susan.kirch@nyu.edu DRL#0918533 Context, Design Decisions, and Preliminary Analysis Rationale for Teaching and Learning Concepts of Evidence The use of evidence is central to scientific knowledge and scientific thinking. Mastery of this cultural tool is essential for developing a scientific worldview. Curriculum design evolution and outcomes • Context • K-8 Elementary and Middle School (enrlmnt. ~800) • School Poverty rate: 75% (NYC-DOE) • Students in temporary housing 2008-2009 (N=13, 2%) • Recent immigrants 2009-2010 (N=24, 3%) • Classroom participants (2009-2010) N=43 • Students receiving special education, (N=20, 47%) • Students receiving ESL services (13, 30%) • Research interview participants (2009-2010) N=31 • Students receiving special education, (12, 39%) • Students receiving ESL services (9, 29%) • Classroom participants (2010-2011) N=126 • Students receiving special education, (22, 17%) • Students receiving ESL services (36, 29%) • Research interview participants (2010-2011) N=106 • Students receiving special education, (16, 15%) • Students receiving ESL services (24, 23%) • STEME Content used to study relationships between • claims and evidence • Cycle 1 (3rd grade): Plant Biology and Seed Storage (historical and contemporary developments in science) • Cycle 2 (3rd grade): Science of Sound (historical developments in science) • Cycle 1 (4th grade): Student choice (physics, astronomy, biology, animal behavior and cognition; contemporary developments in science from EurekAlert; Henry Fountain’s Observatory column) • Cycle 2 (4th grade): Colony Collapse Disorder case study (contemporary developments in science) • Assessments Piloted • How do students use and understand evidence? • Nature of evidence interview (TOC) • Use of evidence tasks (TOC) • Scientific thinker identity interview (TOC) • Interaction Analysis of video (O,TOC) • Do students understand module materials? • Are they interested and engaged? • Unison Reading Protocol (O, TOC) • Running records during unison reading (O) • Feasibility interviews (O, TOC) • Interaction Analysis of video (O, TOC) • What have students learned about scientific evidence? • How can module materials be improved? • Pre-module assessments repeated (see 2) • Interviews with teachers (O, TOC) • Interviews with students (O, TOC) • Quality of module survey/interview (O) • Interaction Analysis of video (O, TOC) • Member checking and co-analysis (O, TOC) • Design Cycle 1a (3rd grade) • Student exploration  Science Conference  Studies of claims and evidence through case studies on the problem of saving seeds in seed banks: Debate of the 2,000-year old seed and the Challenge of Storing Seeds Analysis Cycle 1a (3rd grade) Students succeeded in producing knowledge claims through investigations and defended their conclusions at the Science Conference. Although studies of claims and evidence were explicit, the next design goal was to involve students in endorsing and seeing the relevance of learning about claims and evidence. Evidence is using observations to form, support, and present an explanation (i.e. conclusion, judgment, theory), often in comparison with alternative explanations Design Cycle 1b (4th grade) Claim and evidence in everyday life  Convincing evidence scenarios  Author claims and evidence  Student wondering and questions leading to student research and debates (a major design breakthrough) Contemporary Questions in Science Analysis Cycle 1b (4th grade) Learning about claims and evidence was clearly accepted as a reasonable learning goal on the first day of activities. Overall, student interest and motivation to identify and evaluate evidence was much greater than in the 1a cycle. The next design goal was to determine how to motivate all students to see the utility of learning about evidence and involve them as researchers of knowledge. Project Goals • 1. To develop instructional tools for teaching and learning concepts of scientific evidence through a design-based research process (e.g., Cobb, 2001). Design Cycle 2 (3rd and 4th grade) Begin by engaging students as researchers and producers of knowledge whose research questions are: (1) when we say we know something, what does that mean? (2) If someone says they know something, how could we explore how they came to know it and why they think it’s true? (a major design breakthrough) What is Knowledge? (all); History of the Science of Sound (3rd); Pollination Crisis! The Missing Honeybee Mystery (4th) Preliminary Analysis Table 1. Pre/post interview cycle 1, 4th grade, N=17 2.To theorize the development of evidence as a cultural tool and utilize an Activity Theoretical framework for studying teaching-learning (e.g., Stetsenko, 2009). 1Purple stars (*) show students whose responses did not shift between analytical categories in the post-interview;Blue stars (*)show students who shifted from one analytic category to another between the pre- and the post-interview (e.g., from describing evidence as a thing to describing evidence as in a relationship with a claim). 3.To determine whether elementary students can learn concepts of scientific evidence when provided a coherent, contiguous course of study of evidence. • At the level of operation (O) • At the level of theory of change (TOC) • Preliminary Findings and Implications • Teachers report students are more highly motivated to persevere through difficult text and challenging questions when they conduct research with a purpose. • 3rd and 4th graders were able to describe relationships between observations, claims and evidence. • Participants’ views of evidence as an object shifted to include evidence as a process. • Participants came to appreciate that evidence is used for convincing others and strengthening a scientific explanation rather than merely for proof under the law. • Participants shifted from seeing authoritative text as their primary source of information to seeing themselves as capable of conducting similar investigations. • Students who produce evidence in support of or against a particular claim (from any source) can experience elementary school science in a profoundly meaningful and purposeful way compared to methods that lack reflection on the process of knowledge production in science. • Current retrospective questions • How do students view evidence and how did these views evolve during the project period? • How do students use evidence and how did their use evolve during the project period? • When do students think to use evidence? How do you decide to trust what you see, hear, or read? Students’ Research Associate Responses Student Responses