Building Evaluative Capability in Schooling Improvement

1. Building Evaluative Capability in Schooling Improvement Theories for Improvement and Sustainability Dr Mei Kuin Lai 1

2. What is a theory for improvement? • A theory is a set of linked ideas to explain something. • A theory for improvement is a set of linked ideas about how to improve valued outcomes. • N.B. Sustainability is conceived as part of a theory for improvement 2

3. Why do we need to evaluate a TFI? • TFI guides an individual’s or group’s practices • TFI will result in school/cluster members choosing some practices over others • Examining TFI • Explicit TFI allows the reasons and assumptions for choosing some practices over others to be evaluated. • Evaluating theories is fundamental to effectiveness because not all theories will contribute equally to desired outcomes. 3

4. The evaluation teams’ TFI N.B. All components are inter-related 4

5. Evaluation team’s TFI (cont.) • Developing and refining a TFI involves continually identifying and solving problems (including checking whether problem has changed) to achieve desired outcomes • This process should lead to a solution that is tailored to the needs of the contexts 5

6. Sustainability • A theory for improvement should include a theory of sustainability for valued outcomes • Sustainability is the process of organisational learning to improve outcomes already achieved (Lai et al., 2009) • Sustainability is NOT planned after an intervention. Needs to be planned from the beginning 6

7. Pre-requisites for sustainability(identified from the TFI process) • Verified improvements in student achievement at desired rate and/or desired level (at least two years in a row, or four data points) • Practices known to address the problem are identified including any modification to practices • Processes used to identify and solve problems and monitor solutions are known 7

8. Components for on-going sustainability • Identify teaching and leadership practices and processes essential to maintaining and creating on-going improvement • Embed practices and processes in schools’ core business as part of a coherent instructional programme • Coherence is a set of a set of interrelated programmes for students and staff that are guided by a common framework for curriculum, instruction, assessment and learning climate and that are pursued over a sustained period (Newmann, Smith, Allensworth, & Bryk, 2001). 8

9. Components (cont.) • Have in place systems and processes to identify new challenges and how it will be acted on. • Engaging in the same practices may not raise achievement if the problem has changed • Have a vehicle to systematically access and test knowledge that the school/ cluster needs to continued improving outcomes e.g., professional learning communities (Seashore-Louis, 2006) 9

10. Evaluating a TFI • Are the factual claims underpinning the theory accurate? • Is the knowledge base for choosing particular theory adequate? • Does the school or cluster reasoning and subsequent actions result in the desired outcomes? • Does the theory unintentionally create problems elsewhere? • Is the theory aligned with what we know to be good practice? N.B. if after several cycles of implementing and evaluating TFIS, problem remains unsolved, then need something different (e.g., external expertise for detailed problem analysis) 10

11. 11