70 likes | 193 Vues
This document discusses the complexities of evaluating change initiatives within university settings, specifically referencing the University of Lincoln's OER Change Programme. It emphasizes the need for a developmental evaluation approach, which incorporates rigorous inquiry into the implementation of innovations. This approach balances both creative and critical perspectives, utilizes diverse data sources, and encourages early and ongoing evaluation. Key takeaways include broadening perspectives on what constitutes evidence, integrating evaluation into the planning process, and considering multiple viewpoints.
E N D
Evaluation and dissemination • University of Lincoln OER Change Programme 22-23 March 2012
Evaluating change • Change initiatives are: • complex • non-linear • emergent • unpredictable • long term A different approach to evaluation is needed…
Developmental evaluation “Developmental evaluation is about rigorous inquiry for development… using data in a meaningful way to inform innovation in progress” (Gamble 2008) • an integral part of the planning and implementation cycle, not a bolt-on activity at the end • balances the creative and the critical, the formative and the summative • uses a range of methods, drawing on both ‘hard’ and ‘soft’ data
Choosing your methods • Developmental evaluation is an approach not a method – it involves choosing the right method for the right purpose • Think about what you are evaluating: • ‘simple’ output (sphere of control) • ‘complicated’ outcome (sphere of influence) • ‘complex’ impact (sphere of interest)
Sources of data • What counts as ‘evidence’ – for what purpose and for whom? • Data from evaluation activities – surveys, focus groups, interviews, case studies etc • Data generated by initiative itself – planning documents, decision logs, network maps, influence wheels, meeting notes, emails etc • Naturally occurring institutional data – NSS, satisfaction surveys, retention and achievement data etc
Three ‘take home’ messages • Take a broad view of what evaluation is and what counts as evidence – if it helps, do it • Build your evaluation in early and often – use your planning tools to help you evaluate and your evaluation tools to help you plan • Look at your initiative from multiple perspectives – those inside the initiative, those outside it, and those affected by it