Assessing the non-academic impact of research in REF2021 Jane Millar

1. Assessing the non-academic impact of research in REF2021 Jane Millar CENTRE FOR EXCELLENCE IN COMMUNITY INVESTMENT RESEARCH SYMPOSIUM 18 JULY 2019 • MANCHESTER Twitter @REF_2021 Email info@ref.ac.uk Webhttps://www.ref.ac.uk/

2. Today: • What is REF2021? • Overview of aims, dates, etc • The expert panels • Assessing non-academic impact in REF • Defining impact • Assessing impact • Some examples in the guidance REF Guidance on Submissions: https://www.ref.ac.uk/publications/guidance-on-submissions-201901/ REF Panel Criteria and Working Methods: https://www.ref.ac.uk/publications/panel-criteria-and-working-methods-201902/

3. REF - overview • Undertaken by the REF team on behalf of the four UK higher education funding bodies • Key purposes: • To provide accountability for public investment in research and produce evidence of the benefits of this investment. • To provide benchmarking information and establish reputational yardsticks, for use within the HE sector and for public information. • To inform the selective allocation of funding for research –ca. £2bn per year. • Timing • Submission deadline: 27th November 2020 • Publication of results: December 2021

4. Expert panels 34 sub-panels working under the guidance of four main panels Main Panel members: chairs of Sub panels, international, user, interdisciplinary Sub Panel members: academics and research users, academics nominated by learned societies; user members by government, third sector, business

5. Main Panel C: Social Sciences 13 Architecture, Built Environment and Planning 14 Geography and Environmental Studies 15 Archaeology 16 Economics and Econometrics 17 Business and Management Studies 18 Law 19 Politics and International Studies 20 Social Work and Social Policy 21 Sociology 22 Anthropology and Development Studies 23 Education 24 Sport and Exercise Sciences, Leisure and Tourism

6. 2021 framework Starred quality profile: 4* world-leading 3* internationally excellent 2* internationally recognized 1* nationally recognized UC- below or not research 25% 60% 15%

7. Defining Impact • 4. For the purposes of the REF, impact is defined as an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia. • Impact includes, but is not limited to, an effect on, change or benefit to: • the activity, attitude, awareness, behaviour, capacity, opportunity, performance, policy, practice, process or understanding • of an audience, beneficiary, community, constituency, organisation or individuals • in any geographic location whether locally, regionally, nationally or internationally. • Impact includes the reduction or prevention of harm, risk, cost or other negative effects.

8. Assessing Impact: case studies • Each submission must include Impact Case Studies • Up to 20 people (FTE) – 2 cases • Add 1 more case study for each 10 FTE • 160 FTE and above – ten plus one for every extra 50 • Key dates • The underpinning research - 1 Jan 2000 to 31 Dec 2020 • The impact assessment period - 1 August 2013 to 31 July 2020 • Approach to impact also in Environment assessment

9. Assessing Impact: reach 288: Reach will be understood as the extent and/or diversity of the beneficiaries of the impact, as relevant to the nature of the impact. Reach will be assessed in terms of the extent to which the potential constituencies, number or groups of beneficiaries have been reached; it will not be assessed in purely geographic terms, nor in terms of absolute numbers of beneficiaries. The criteria will be applied wherever the impact occurred, regardless of geography or location, and whether in the UK or abroad.

10. Assessing Impact: significance 289: Significance will be understood as the degree to which the impact has enabled, enriched, influenced, informed or changed the performance, policies, practices, products, services, understanding, awareness or wellbeing of the beneficiaries.

11. Assessing Impact: overall 290: The sub-panels will make an overall judgement about the reach and significance of impacts, rather than assessing each criterion separately. While case studies need to demonstrate both reach and significance, the balance between them may vary at all quality levels. The sub-panels will exercise their judgement without privileging or disadvantaging either reach or significance.

12. Impact – evidence 231. Each case study must provide a clear and coherent narrative that includes an account of who or what audiences, constituencies, groups, organisations, places, publics, sectors and so on, have benefited, been influenced, or acted upon. The sub-panels will use their expert judgement regarding the integrity, coherence and clarity of the narrative of each case study, but will expect the key claims made in the narrative to be supported by evidence and indicators. 309. The panels anticipate that impact case studies will refer to a wide range of types of evidence, including qualitative, quantitative and tangible or material evidence, as appropriate. ... A diversity of evidence is welcome, and no type of evidence is inherently preferred over another. 310. Testimonials should draw on statements of fact and relate specifically to the impact(s) claimed. ….Where testimony is cited, it should be made clear whether the source is a participant in the process of impact delivery (and the degree to which this is the case), or is a reporter on the process.

13. Impact – selected example Areas Types Indicators

14. Impact – selected example

15. Discussion How can we increase the impact of research in community investment that delivers better outcomes and responds to the Research Excellence Framework?