The Team – NHMRC Partnership Grant

1. A Logic Model Tool for Evidence Informed Policy MakingDr Peter Orpinfor the NHMRC Partnership Grant Team

2. The Team – NHMRC Partnership Grant • Industry Partners • Department of Health and Human Services • Department of Premier and Cabinet • Chief Investigators • Prof. Judi Walker – Monash University • Dr Peter Orpin – University of Tasmania • Prof. John Humphries – Monash University • Dr Matthew Carroll – Monash University • Kim Boyer – University of Tasmania • Associate Investigators • Prof Helen Bartlett – Monash University • Prof. Andrew Robinson – University of Tasmania • Mr Brendon Davidson – Department of Health & Human Services www.utas.edu.au/ruralhealth

3. The Project Initial Aim: To develop a tool to support policy makers to increase the evidence base of rural ageing well policy and practice. An evolution: To a more generic tool to facilitate evidence informed policy making – using a rural ageing well case example: • Very limited specific, quality ageing-well intervention evidence let alone rural ageing well evidence • Considerable, more generic, QOL support intervention evidence but dispersed widely across disciplines, professional practices and literature and, therefore, difficult to find, collate and operationalise • Issues around evidence proliferation, dating, translation and incorporation into practice • Restricted translation capacity of social science evidence What appeared to be needed was not a packaged suite of rural ageing well evidence but a tool to ‘sensitise’ policy makers to the potential need for evidence and to facilitate their sourcing of that evidence. www.utas.edu.au/ruralhealth

4. Evidence and The Policy Process • Continuing strong support for increased use of evidence in policy, however: • Evidence based policy making (EBPM) recognised as flawed both empirically and normatively SANDERSON, I. 2011. Evidence-based policy or policy-based evidence? Reflections on Scottish experience. Evidence & Policy: A Journal of Research, Debate and Practice, 7, 59-76. • Not exactly ‘muddling through’ (Lindblom 1959) - some (variable) rational linear evidence based elements - but also ‘values, interests, personalities, timing, circumstances and happenstance’ BANKS, G. 2009. Evidence-based policy-making: What is it? How do we get it? ANZSOG/ANU Public Lecture Series P.4 • Not an event but a social dialogue/discourse – ‘a formal [and perhaps informal] struggle over ideas and values.’ RUSSELL, J., GREENHALGH, T., BYRNE, E. & MCDONNELL, J. 2008. Recognizing rhetoric in health care policy analysis. Journal of Health Services Research & Policy, 13,40-46.p. 40 • Role for evidence in informing and influencing discourse ‘change[ing] mental models and minds DAVIES, H. T. O. & POWELL, A. E. 2012. Communicating social research findings more effectively: what can we learn from other fields? Evidence & Policy: A Journal of Research, Debate and Practice, 8, 213-233. • Best hope is evidence informed policy processes (EIPM) www.utas.edu.au/ruralhealth

5. Facilitating EIPM • Policy-makers receptive if it assists them to do their job LOMAS, J. & BROWN, A. D. 2009. Research And Advice Giving: A Functional View of Evidence Informed Policy Advice in a Canadian Ministry of health. The Millbank Quarterly, 87, 903-926. • However, also poses disincentives – can make life more difficult BANKS, G. 2009. ibid • EIPM theorisation and strategizing largely ‘push’ focussed– researchers seeking to understand the policy process from the outside in order to identify openings, barriers and opportunities to push evidence into the process (see BELL, E. 2010. Research for Health Policy, New York, Oxford University Press for an example). • Much less attention to policy maker evidence ‘pull’ – that is, what prompts policy makers to actively seek out evidence BANKS, G. 2009. ibid • ‘Evidence pull’ starts with policy makers having questions that need answers which, in turn, has its roots in a critical reflective approach to policy making A revised project aim:a tool that assists policy makers to identify those elements within the process that require critical examination and testing through further information (including evidence) seeking www.utas.edu.au/ruralhealth

6. A Logic Model Based Approach • Program Logic Modelling already well established within program evaluation as a tool for making program assumptions explicit and open to testing through information seeking.W.K. KELLOGG FOUNDATION 2004. Logic Model Development Guide: Using Logic Models to Bring Together Planning, Evaluation, and Action. Battle Creek, Michigan: W.K. Kellog Foundation. • The generic program logic model looks something like this: • As generally operationalised it is used to identify and target some critical points at which the performance of a program may be tested: • Were the inputs or resources adequate? • Did the activities take place as intended? • Did they produce the outcomes as expected? • It is also often notable for lack of questions addressing underlying assumption Logic modelling, however, has potential utility at all stages of the policy or program development process, from initial planning to post hoc evaluation. www.utas.edu.au/ruralhealth

7. The ‘if we, then’ logic Program logic models have at their heart one, or more, ‘if we, then’ logic models – that is, if we put these resources into these actions then we will achieve these outcome Such causal linkage models underlie and shape policy discourses and their ‘struggles over ideas and values’ – but are rarely explicitly modelled, particularly at any level of detail. ‘If we, then’ logic implies a range of assumptions – usually implicit and unreflected upon - about current states and the associated causal mechanisms either underlying those current states or the achievement of change within those states www.utas.edu.au/ruralhealth

8. A logic model allows those assumptions to be made explicit and facilitates their testing through critical reflection and application of information • The nature of the ‘problematic’ state – and its root causes • The range of possible pathways from the problematic to the unproblematic • The appropriateness, feasibility and actual implementation of the intervention(s) • The mechanisms by which these interventions might have their effect • What exactly a more desirable state would/should look like and acceptable alternatives • The (potentially changing) environment(s) in which this is all taking place www.utas.edu.au/ruralhealth

9. By formally modelling the policy or practice aspiration we: • Make all assumptions and positions fully explicit and open to testing • Precipitate critical reflection and discourse around these issues • Increase the imperative to seek clarifying information (evidence ‘pull’) whether that be formal evidence, informal intelligence or a discourse around values and ideas. • Provide an anchoring or re-orienting point to return to when the discourse loses its way or hits an impasse. www.utas.edu.au/ruralhealth