1. The Research Excellence Framework 23 Mar 2010 Graeme Rosenberg REF Manager

2. Consultation feedback • Next steps • Impact and the pilot exercise

3. Consultation feedback (1) • Overwhelming support for REF to be a process of expert review, with output quality the dominant factor • Support for institutional selection of staff, with tighter eligibility criteria • 3 or 4 outputs per person? • Cautious support for citation data to inform the review of outputs, in some UOAs

4. Consultation feedback (2) • Broad support for including ‘historic’ impact in the REF, subject to the development of a robust method and an appropriate weighting • Objections appeared to reflect concerns about ‘predicting’ impact • Support for a wide definition of impact, long time-window and a case study approach • Much constructive feedback about the challenges of defining, evidencing, attributing and assessing impact

5. Consultation feedback (3) • Support for including ‘environment’, using a template, and desire for a more even weighting with impact • Should REF use HESA data for PGRs and research income? • Support for four main panels and fewer sub-panels, though: • The proposed panels too large/diverse in some cases • Some reconfigurations suggested in the humanities • Will multiple submissions be allowed? • A coherent environment should not be expected

6. Consultation feedback (4) • What more can be done to support interdisciplinary research? • Concern about the timetable • Support for the equalities measures • Acceptance of burden and unwillingness to cut this at the expense of robustness

7. Next steps • Boards of the UK funding bodies considering the consultation outcomes • Impact pilot exercise is well underway • An early start to recruiting Main Panel Chairs Designate • Establishing an expert group to advise on equalities and diversity in the REF • Decisions on the weighting and method for impact to be taken after the pilot concludes (autumn 2010)

8. Parameters for assessing impact • Wide definition of impact

9. Parameters for assessing impact • Wide definition of impact • Impacts that have occurred during the REF period, underpinned by research over a longer timeframe • Impact of the unit as a whole, drawing on exemplars • Impact underpinned by excellent research, of all types • Qualitative explanation supported by quantitative indicators • Assessment through expert review, within panels according to appropriate criteria • Input from external audiences, users and beneficiaries in partnership with academics • Minimum burden necessary to enable robust assessment

10. The impact pilot exercise • 29 UK HEIs, each submitting to two UOAs • Impact during 2005-09, underpinned by research since 1993 • Each submission to include: • An overview statement • One case study per 10 research staff • Assessment and feedback by expert panels Medicine Earth Systems & Environmental Science English Language & Literature Physics Social Work & Social Policy

11. Some questions • Who are the audiences, users and beneficiaries? • What are the pathways to impact? • What kinds of impact have arisen? • How robust is the evidence? • What kinds of indicators are useful? • What criteria differentiate between case studies?

12. The impact pilot panels • Clinical Medicine • Chair: Alex Markham • NHS, NICE, MHRA, ABPI, GSK, Roche, INVOLVE, GMC, ARC, BHF • Physics • Chair: Peter Saraga • Rolls Royce, IBM, Vodaphone, DSTL, TSB, STFC, NHS, NI Science Park, Science Museum Earth Systems & Environmental Science • Chair: Bob Watson, CSA DeFRA • BP, Shell, Willis Re, Viridor Waste, RSPH, Scottish Government, Environment Agency, Natural England, Greenpeace, DfID

13. The impact pilot panels • Social Work & Social Policy • Chair: Paul Wiles, CSA Home Office • BAAF, AgeUK, SCIE, National Youth Agency, Nuffield Foundation, Hanover Housing Group, EHRC, Local Government Association, Scottish Government, DWP • England Language & Literature • Chair: Judy Simons • BBC, Authors/journalists/reviewers, British Library, British Council, Shakespeare’s Globe, Penguin Press, Brunswick Group LLP, AHRC

14. Thank you for listening www.hefce.ac.uk/ref ref@hefce.ac.uk