Q uestions, Claims and Evidence: Teaching Argumentation in the NGSS through the use of a Science Writing Heuristic

1. Questions, Claims and Evidence: Teaching Argumentation in the NGSS through the use of a Science Writing Heuristic Brian Hand University of Iowa

2. Discussion format • Theoretical perspectives • Randomized field trail results • Classroom aspects

3. Next Generation Science Standards Scientific and Engineering Practices 1. Asking questions (for science) and defining problems (for engineering) 2. Developing and using models 3. Planning and carrying out investigations 4. Analyzing and interpreting data 5. Using mathematics and computational thinking 6. Constructing explanations (for science) and designing solutions (for engineering) 7. Engaging in argument from evidence 8. Obtaining, evaluating, and communicating information

4. Science • The advancement of science is about a process of construction and critique • Scientist negotiate with each other • Scientists do not advance science through information transfer – if this was the case who gave the first scientists the information to pass on? - who gives the current generation of scientist the “new” knowledge

5. Science Argument • Is a central component of science • Requires participants to negotiate meaning both publicly and privately • Is bound by a structure linking questions, claims, evidence and rebuttals • Constructed knowledge is tested against nature

6. Our working definition • Argument Based Inquiry is inquiry that is intended to build students grasp of scientific practices while motivating an understanding of disciplinary big ideas. Construction and critique of knowledge are centrally located through an emphasis on the epistemological frame of argument by engaging them in posing questions, gathering data, and generating claims supported by evidence.

7. This perspective of argument builds on the work of Walton who suggests that argument is a logical contribution to the resolution of unsettled knowledge. This more general perspective on argument is valuable as it recognizes the use of argument as a learning tool; thus the immersion of students in argument throughout their inquiry.

8. Douglas WaltonUniversity of Toronto • Argument • Deals with unsettled knowledge • Trying to persuade others • Explanation • Deals with settled knowledge • To inform others

9. Argument • Made up of questions, claims and evidence • What is a claim? • What is evidence? • What is the relationship between these elements?

10. Connections

11. Argument • Deals with questions, claims and evidence • There must be connections between questions, claims and evidence • There has to be strong coherence between the various components • Arguments require reasoning – not something to be simply learned

12. Critical Issues that need to be engaged with • We do not pay enough attention to ideas such as data, evidence, explanation • Researchers use such ideas – as published in articles • Data/evidence • Claims, evidence, reasoning • Evidence and explanations

13. Data and evidence • Is this distinction important? • Students have trouble separating these two • “Data does not speak” • We have to do something with data to get to evidence

14. Evidence and Reasoning • If we have to do something to data to get evidence – what is it? • Critically we have to reason about the data – we have to make critical decision about • What data points to use? • Are there patterns? • If we remove reasoning from evidence we have data

15. Relationship

16. Evidence and explanation • Simply question – if evidence is not an explanation – what is it? • Is not evidence a reasoned explanation about particular data points and how they fit together? • All evidence is explanatory, but not all explanations are evidentiary

17. Two essential components of science • Language - there is no science without language - means that we have have to pay attention to all the different discourses/representations associated with science • Argumentation - is a critical process that is central to the way in which science knowledge is constructed

18. The Science Writing Heuristic approach is based on earlier Halliday work (70’s) You learn about language while you learn through using language • Means that students learn about argument while they learn through using argument

19. Importance? • These distinctions are not trivial • There is a distinctly different orientation to the learning of argument based inquiry • Is it something “done to” students or something students should “be immersed” in?

20. The Science Writing Heuristic Templates Teacher’s template • Exploration of pre-instruction understanding • Pre-laboratory activities • Laboratory activity • Negotiation I - individual writing • Negotiation II - group discussion • Negotiation III - textbook and other resources • Negotiation IV- individual writing • Exploration of post-instruction understanding Student’s template • Beginning questions or ideas What are my questions about this experiment? • Tests and Procedures What will I do to help answer my questions? • Observations What did I see when I completed my tests and procedure? • Claims What can I claim? • Evidence What evidence do I have to support my claim? How do I know? Why am I making these claims? • Reading How do my ideas compare with others? • Reflection How have my ideas changed?

21. Randomized Field Trial • Involves 48 grade 3-5 buildings in Iowa • 24 treatment, 24 control • Divided into 5 clusters within the state • 8 days of inservice – 5 in summer, 3 during the year • Follow up monitoring within school • Collection of teacher video – one per semester/year • Collection of Iowa Test of Basic Skills/Iowa Assessment data • Implementation of Cornell Critical Thinking test pre/post at grade 5 level

22. Critical Thinking Improvement Scores Year 1

23. Critical Thinking Improvement Scores Year 2

24. Effect sizes for each cluster

25. Transfer – what do we mean Domain Specific Knowledge Domain General Knowledge Transfer Cornell Critical Thinking Test Argument-based Inquiry

26. Data Used • The data used for the following analysis' are the paired 3rd - 5th grade and paired 4th - 6th grade national standardized scores on the ITBS and Iowa Assessments for school year pairs 2006-07 with 2008-09, 2007-08 with 2009-10, 2008-09 with 2010-2011, and 2009-10 with 2011-12. • The data is paired by student • The associated demographics are also used

27. Equivalence of ITBS and Iowa Assessments • ITBS and Iowa Assessments share three sections that were taken by all schools in the study: Reading, Mathematics, and Science. • The Math I and Math II Scores from the ITBS relate with the Math Comprehension score from Iowa Assessments • For each combination of subject, grade, and year the National Standardized Scores were standardized. For the ITBS Math Scores they were added, then standardized

28. Mixed Models • For each of the four groups of students as described previously, for each of the three test types (RC, M, and SC), mixed models were fit • The predicted value was the change in test score (DRC, DM, or DSC) Note: Scores standardized as previously mentioned • The base-level fixed effects were pretest score (RC, M, or SC), ASN, BLK, HSP, FRL, ELL, GAT (All students only), SED (similarly), and TRT

29. Mixed Models Continued • The interactions included in the models were TRT with all the other fixed effects. • The two random effects included in the modes were UNIT (the unit for which treatment or control was assigned, usually a school building) and DIST (the block of units used to control variability in the assignment of treatment and control) • For each model non-significant variables were removed (t-score < 1.66 (relating p-value .1)) unless they were base fixed effects who had a significant interaction term or the TRT fixed effect.

30. All Students - Reading • DRC ~ RC + TRT + SEM + DSEM + ASN + BLK + SED + GAT + FRL + ELL + (1|UNIT) + (1|DIST) • Notes • TRT - Weak Positive (t = 1.66) • No significant interactions

31. All Students - Mathematics • DM ~ M + TRT + DSEM + GEN + ASN + BLK + HSP + SED + GAT + FRL + ELL + TRT:GEN + TRT:BLK + TRT:SED + TRT:GAT + TRT:FRL + TRT:ELL + (1|UNIT) + (1|DIST) • Notes • TRT – Very Very Strong Positive (t = 11.23) • TRT:GEN – Weak Negative (t = -2.03), TRT:BLK – Weak Positive (t = 1.97), TRT:SED – Very Strong Positive (t = 3.44), TRT:GAT – Very Very Strong Negative (t = -7.34), and TRT:ELL – Weak Postive (t = 2.10)

32. All Students - Science • DSC ~ SC + TRT + DSEM + GEN + BLK + HSP + SED + GAT + FRL + ELL + (1|UNIT) + (1|DIST) • Notes • Not significant TRT • No significant interactions

33. All Students - Comments • There is small evidence to support students do better with Reading Comprehension in the Treatment group • There is very strong evidence to support that students do better with Mathematics in the Treatment group • The female disadvantage is reduced with treatment • The African American disadvantage is reduced with treatment • The special education disadvantage is reduced with treatment • The gifted and talented advantage is reduced with treatment • The free and reduced lunch disadvantage is reduced with treatment • English language learners in treatment are as well off as non-English Language Learners in the Control group • There is no evidence to support improvement is Science Comprehension associated with Treatment

34. Special Education Conclusions • There is no evidence to support that treatment leads to a change in Reading Comprehension • Males receive an advantage in Reading Comprehension with treatment • Free and reduced lunch students receive a further disadvantage with treatment • There is very strong evidence to support that treatment leads to an improvement in Mathematics • There is no evidence to support that treatment leads to a change in Science Comprehension • Free and reduced lunch students receive a further disadvantage with treatment

35. Gifted and Talented Conclusions • There is no evidence of an effect of Treatment on Reading Comprehension • There is weak evidence to suggest a negative effect of Treatment on Mathematics • The female disadvantage is reduce in the treatment group to the point were a female in the treatment group is better off then in the control group • The free and reduced lunch disadvantage is reduced in the treatment group to the point where a FRL student in the treatment group is better off then in the control group • There is no evidence of an overall effect of Treatment on Science Comprehension • The Hispanic advantage almost disappears in the treatment group

36. Traditional Conclusions • No evidence of an overall effect of Treatment on Reading Comprehension • Treatment seems to decrease the Asian Advantage • Treatment seems to decrease the free and reduced lunch disadvantage • There is strong evidence of an improvement of Mathematics scores because of Treatment • An African American in the treatment group has the same advantage as a Caucasian in the control group • An English Language Learner in the treatment group is better off then a non-ELL in the control group • A free and reduced lunch student in the treatment group is almost as well off as a non-FRL in the control group • No evidence of an overall effect of Treatment on Science Comprehension • The Asian advantage decreases with the Treatment group

37. Classroom Conditions • How well do teachers implement? Is there a difference between treatment and control? • Do they have more science teaching in school? • Is there a carry over to other subjects? • Can we see a shift in how the classroom environment looks? • Does this impact language events?

38. Teacher Implementation ScoresYear 1

39. Number of lessons per week

40. Minutes/science lesson

41. Transfer of approach into other disciplines

42. Classroom environment

43. Teacher-student talk

44. Writing • Does implementation matter? • Do we understand what is happening?

45. Sources of meaning Complexity of Reasoning Diagrammatic Representation Developing Single Reasoning Personal (Intuitive) Contextualized/ Perception-based Fuzzy Understanding Single Reasoning Alternative Explanation Developing Chain of Reasoning Comparing Ideas Chain of Reasoning Consolidating Ideas Developing Reasoning Network Further Negotiation Reasoning Network Coherence Scientific (Reflective)

46. Research funded by a grant from the US Department of Education through the Institute of Education Sciences, award number R305A090094-10.