Data Driven Design for Learning

1. Data Driven Design for Learning Teaching, Learning and Standards Division June 2008

2. Session Organisation 0800 – 0845: Overview and framing the inquiry 0845 – 1000 and 1030 – 1130: Workshops

3. Changing teacher practice and knowledge Design Inquiry Questioning Evidence based Reflection and Evaluation Rules and Procedures Prescriptions Steps

4. 1.Knowledge about content 2. Knowledge about learning and learners 3. Knowledge aboutteaching 4. Pedagogical content knowledge Setting the contextDescribing Expert Teachers System context and effectiveness factors School context and effectiveness factors Teacher Professional Characteristics Sources: Hattie 2003, Killen 2006, Hill 2003

5. Cueing and retrieving knowledge Think of a program, initiative, typical school or teacher support, materials development that you currently are focused on. Where is the starting point in relation to the columns below? Where does this lead to in relation to the four columns? What questions are considered in each of these columns?

6. Comparing existing knowledge with research frames The 8 Learning Management Questions (Lynch, 2007)

7. Planning for Teaching

8. Planning for Assessment

9. Design for learning using the 8 Learning Management Questions (Lynch, 2007)

10. Design for learning using the 8 Learning Management Questions

11. Design for learning using the 8 Learning Management Questions (Lynch, 2007)

13. What impacts on student learning? What are the data requirements? NTCF Reports (CeTool/BIC) Demographics MAP/NAPLAN reports Attendance School perception surveys School Programs Perceptions ELA Schools-LLANS Student Learning

14. A system is not the sum of its parts, but rather, the product of the interaction of the parts Russell Ackoff

15. Lines of Inquiry Improvement Progress Accountability Achievement

16. Why use data Design for learning Monitoring impact of professional learning Close the achievement gap APIF Process • Annual Operational Plans > data analysis to determine short term goals and targets • School Performance Reports • 4 year Strategic Improvements Plan Performance Review

17. Using data to monitor the impact of teacher learning and development

18. Data to close the achievement gap Ruth Johnson (2002) Using data to close the achievement gap: How to measure equity in our schools in Holcomb, 2004, Getting Excited About Data Schools with large populations of low income students and students of color mitigate perceived achievement barriers by: • Continuous inquiry and monitoring through the use of data • High goals, standards, expectations and accountability for students and adults • Well qualified and culturally competent teachers • Curriculum content and rigour Timperley, Wilson, Barrar, and Fung, 2007, Teacher Professional Learning and Development

19. Design for learning using the 8 Learning Management Questions (Lynch, 2007)

20. How? - Process Analysis Data Browse Define expected levels Line of Inquiry into data sources Analysis Data Delve Interpretation and Goal Setting Plan evidence based action

21. Example: Thinking through data analysis flowchart Define expected levels for this stage or year level of schooling What does perception data tell us about students, teachers and parents? (perceptions+ demographics) Look at data for stage/ Year level for past three years (student learning) How are we teaching ____ at this stage/ Year level of schooling? (school processes by demographics) How many students are at, below, or beyond expected levels? (student learning + demographics) How are these data related? Disaggregate data by demographics to understand what groups of students are at, below above expected level? (student learning + demographics) (Bernhardt, 2004:170)

22. Defining system expected levels

23. Defining system expected levels

24. What do National Benchmarks look like for ESL Learners?

25. Design for learning using the 8 Learning Management Questions (Lynch, 2007)

26. Influences on student learning Using meta-analysis (synthesis) research > impact measured by effect sizes .40 .30 .50 .60 .15 Typical Teacher Effects .70 .80 Developmental Effects ZONE OF DESIRED EFFECTS 0 .90 REVERSE 1.0

27. Robert Marzano’s 1997 meta-analysis instructional strategies

28. John Hattie’s meta-analysis on what makes a difference to student learning (2003) 0.80 + Large effect 0.40 – 0.80 Moderate effect

29. A teaching and learning framework based on evidence 1. Learning is enhanced when learning opportunities are tailored to individuals’ current levels of readiness 2. Learning is more effective when it leads to deep understandings of subject matter. 3. Learning is more effective when learners are supported to monitor and take responsibility for their own learning.

30. Vivianne Robinson’s meta-analysis on leadership (2007)

31. NT Teaching and Learning Framework • Positive Attitudes and Perceptions through • Classroom Tasks • Classroom Climate • Thinking and Learning Habits • Thinking on my own • Thinking together • Engaging With • and Acquiring • Knowledge • Link to prior learning • Explicit teaching • Immersion • Construct meaning • and models • Using and Connecting • Knowledge Meaningfully • Complex reasoning processes • Open ended • learning opportunities • Connecting to world beyond • classroom • Practicing and • Deepening • Knowledge • Organise and store • Shape and Internalise • Complex reasoning processes • Questioning • Reflecting • Sustained Dialogue

32. Design for learning using the 8 Learning Management Questions (Lynch, 2007)

33. Assessment Design PrinciplesMasters and Forster, ARK Developmental Assessment Fairness Are the tasks constructed, presented and conducted so that all students have an equal chance of demonstrating their learning against the NTCF? Do the assessment tasks require specific knowledge or skills which some students may not have, and which have not been explicitly taught? Curriculum relevance Do the tasks represent all the valued learning you want the students to have undertaken? Do they assess a sample of all the important aspects of the learning? Assessment Task Instructional utility How useful is the method in yielding insights into students knowledge and conceptual understandings and can be used to guide teaching and learning? Reliability and comparability How accurately does the collected evidence reflect what the learner knows and can do? Practical Convenience Is the method and task easily incorporated into classroom activities?

34. AssessmentFORlearning: occurs when teachers use inferences about student progress to inform their teaching. Assessment AS learning: occurs when students reflect on and monitor their progress to inform their future learning goals. How will we know when learners have achieved the learning outcomes? AssessmentOFlearning:occurs when teachers use evidence of student learning to make judgements on student achievement against goals and standards. DET Victoria, Assessment Professional LearningModule 5: Making Consistent Judgments

35. What? Lines of Inquiry into data

36. What? Data Sources English - Writing Most useful to the system and school Least useful to the system Least useful to the classroom teacher Most useful to the classroom teacher

37. What? MAP Reports- Writing

38. Example: Primary School A Summary by Criteria

41. Gaps • Vocabulary • Cohesion • Punctuation

42. NTCF Data

43. Now What? CeTool planning using monitoring data

44. Business Intelligence Centre – NTCF and MAP Reports Data Analysis and Reporting Tool Log On: nightcliffsch-sec and Password: q1w2e3r4 MAPRET Log On: nightsch-sec and Password: 371346

45. Frequently Used NTCF Reports

46. So What? If you don’t know where you are going anything you do will get you there Anonymous in Bernhardt, 2004

47. Defining system expected levels

48. Defining system expected levels

49. What do National Benchmarks look like for ESL Learners?

50. NT DEET System Expected Levels