What do student views on teaching excellence mean for recruiters?

1. What do student views on teaching excellence mean for recruiters? Jim Dickinson, Chief of Staff, UEA Students’ Union

2. Plus:The student view on Value for Money(and what might happen next) Jim Dickinson, Chief of Staff, UEA Students’ Union

3. About us • Group of 40 Students’ Unions from around the diversity of the sector • Interested in student views, research, and development of HE policy within HEIs and nationally • Not NUS but works with it • National thematic studies (SU rpn, TE, VFM, Accomm) • Evidence gathering (FOIs, Projects and Practice)

4. Introduction and background • Creation of Teaching Excellence Framework and rapid flux in scope, depth and definitions/weightings • Shift in assessing/assuring/determining HE quality from processes/voices to metrics/outcomes • New Office for Students; regulatory framework done now TEF development (and NSS) • TEF as a tool for student choice and a driver of change • Student voices absent from much of the debate about TEF and OfS

5. Introduction and background • Research to look at student views of teaching excellence • What is it? • How/should accountability work? • What would assessment do to decision making? • Survey with quantative and qualitative • 10k students, 124 Universities, Last Summer, All year groups • Weighted for institution and representative of UG and PGT cohort • Results for SUs as well as national results • This presentation looks at the headlines along with key questions and conclusions

6. Should the Government be running an exercise to encourage excellent teaching in universities?

7. Teaching which takes into account the needs of it's student, is responsive, innovative and supportive teaching to deliver competent graduates true learning not just memorising for the test or a grade Where students feel that their teachers best interests align with their own best interests, and both parties encourage eachothers performance. Making students have a real interest in the subject whilst also giving them a high proficiency in . Providing a student with all of the means to achieve their individual targets during their course of studies. It means to teach students the real life experiences before graduating to fully support them and give them hope to achieve success in their lives. Teaching that inspires you

8. Judging a university’s teaching quality % of students that either ‘Agreed’ or ‘Strongly Agreed’ with assessing teaching quality on each of the mentioned factors

9. Factors that indicate that a university has excellent teaching

10. Most important factors for assessing Excellent Teaching

11. Least important factors for assessing excellent teaching

12. University resourcesoverview % of students that considered each factor as either ‘Important’ or ‘Very Important’

13. Assessing Teaching Excellence • There is strong support amongst students for a Government exercise that encourages excellence in teaching (85% agree) • Students have a wide understanding of teaching excellence and define it in multiple ways that the TEF does not define • Students believe that TE should encompass a number of factors related to the teaching and learning environment not currently in TEF (86% IT, 93% Library, Course Resources 93%) • There is less support amongst students for employment metrics being in the TEF than other factors (1 in 4 do not agree they should feature but 90% agree that quality of teachers should be included)

14. Should universities be held to account if students are dissatisfied with their experience?

15. Should universities be held to account if students drop out?

16. Should universities be held to account if their graduate jobs ratings are poor?

17. Should universities be held to account if the teaching isn't good enough to enable you to succeed?

18. Who should hold Universities to account on satisfaction? Government A national students' council and a Parliamentary committee The Chancellor (Each university is responsible for their own success) Student Reps, Union and by an independent university regulator Panel of faculties within the university MI6 The Govt through an Ofsted-type organisation

19. What should be the consequences? Discounted repayment loans A clear outline for improvements and development, set out through an agreement between students (possibly through the student union) and the university's governing board The Govt through an Ofsted-type organisation Less people would go there for the bad rep already surely? Refunds/Rights to Students - Like Consumer Protection Panel Have to address feedback and make active plans to Improve the quality of teaching from feedback.of faculties within the university Impact on lecturers' performance reviews, negative publicity (e.g. through publicly reported outcome measures), ultimately funding reduction (although this is usually counterproductive)

20. Accountability • Whilst students agree that Universities should be held to account for teaching “not good enough to enable them to succeed”, only 34% agree they should be held to account if graduate jobs ratings are poor, and just 18% agree they should be held to account if students drop out. • Students are highly conflicted on what should happen if satisfaction rates are poor- some favouring public shaming (with the threat of poor recruitment a driver for improvement) and others more traditional improvement methods • Students are also conflicted on who should hold Universities to account- with a clear divide between external approaches (ie regulation/government) and internal approaches (ie other academics, SU)

21. Do you agree with the government giving ratings (Gold/Silver/Bronze) to universities based on teaching excellence?

22. Should student fees be linked to the rating of the university?

23. Should fees be linked to TEF? • No - Course may be high quality even if the university as a whole isn’t. • Value for money for students, could deter students from lower socioeconomic status. • Will put off some people going to better universities and not achieving their best potential • You should not have to pay more for education filters should be based on merit. Eg raise the minimum requirements of the good university. But fees should be standardised or ideally scrapped as an investment into higher education is an appreciating investment in the countrys' future. • It is massively unfair that students should be made to pay more for a good standard of teaching, it should be a universally good standard as increases sets students from disadvantaged background up for failure and more debt despite their intelligence. • I would assume higher rated uni have higher entry grades so i dont see why, after proving your ability, you would then be blocked by a higher price barrier

24. If your university had been given a Gold rating when you applied, would it have affected your decision to apply?

25. If your course had been given a Gold rating when you applied, would it have affected your decision to apply?

26. If your university had been given a Bronze rating when you applied, would it have affected your decision to apply?

27. If your course had been given a Bronze rating when you applied, would it have affected your decision to apply?

28. “Teaching may not be equal across different programmes. You could end up paying more to go to a uni that has achieved gold status by teaching means that aren't necessarily carried into the subject you aim to study there “Discourages poorer people from going to better universities “Because this will price out poorer students from going to good universities, and encourage students to attend a poor university due to cost “It creates an unfair system, where by less well off students may be discouraged from applying to high ranking universities. “In any case 3 silly ratings is hardly sufficient to categorise a university that works in numerous fields. For example some bronze rated universities may in fact excel in a particular field, more so than a gold rated university.

29. Psychology • Whilst only around 1 in 5 disagree with “Gold, Silver, Bronze” rankings, 4 in 5 don’t agree that student fees be linked to the rating of the university. • Students regard charging more for a “better” university as faulty or unjust • When considering factors that indicate that a university has excellent teaching, students are over three times less likely to identify high graduate earnings when compared to access to resources. • 48% of students would have reconsidered or not applied to their University if they had known it was rated “Bronze” • 7% of students would have reconsidered or not applied to their University if they had known it was rated “Gold” • Similar proportions would have reconsidered or not applied if the same judgements were levied at course level.

30. If I were to have an academic problem I feel I have access to independent advice

31. Key Conclusions • Students want a framework and are happy with Universities being held to account • Students are happy that this includes factors directly related to the teaching and learning experience • They are divided on who and how accountability should work • Students are much less supportive of impact measures such as drop out or graduate employment being used as a proxy for “teaching Excellence” • Students overwhelmingly disagree with linking fee levels to the exercise • They regard it as unfair or unjust esp from a widening access POV or debt burden POV (tax on talent) • Given a large proportion of students would have reconsidered their choice if rated Bronze there are CMA/Consumer rights issues if a Bronze course was run in a “Gold” institution • There may be unintended consequences in the rankings with the potential that a Gold ranking would have put some students off from applying to a particular university/course

32. We found no evidence that students understood that the TEF Medals are based upon performance against a benchmark rather than absolute performance: whenever students talked aboutmedals, they consistently assumed that a ‘Gold’ rating at one institution is directly comparable to the ‘Gold’ rating at another institution.

33. The student view on Value for Money(and what might happen next) Jim Dickinson, Chief of Staff, UEA Students’ Union

34. Value for Money • Consistent feature in SU surveys and elected officer manifestos • There are “big debates” about value, consumerism, fees, etc but we wanted to hear from students and focus in on their perspectives/concerns • Student interests are not always aligned with “taxpayers” interests • What if we asked about VFM regardless of wider debates, fees review etc

35. About the study • Led by two Students’ Unions • 31 Students’ Unions collectively developed lines of enquiry/hypotheses • Carried out by Trendence UK • 31 SUs involved in scoping and design, 5,685 current HE students in England (plus a Scottish sample), 534 recent graduates, 410 school students (in year 12 or 13), 133 different providers, FTUG, PTUG, PGT, PGR all represented, weighted by provider and by gender and quota sampling used • Commissioned by OfS for SP and VFM agenda • Key questions now are how should providers be nudged or required to respond?

36. What should be done? Some elements are extrinsic to this sphere • Society attitude/ideology • Socioeconomics, student profile, course type • Overall funding model and fees review This means that: • Some elements could be/are regulatory (OfS require) • Some elements are within providers’ sphere of influence and control (OfS nudge)

37. Subsidies- Cross and Student What should fees fund? • Students are particularly interested in cross-subsidies. • ‘Tuition’ fees as a misnomer: departments and services (one for the DfE review!) What should be subsidised? • Direct benefit- highly positive • Indirect benefits- lots of “NAND”s • “Proper” cross subsidy- highly negative • Underpinning issue is that whilst the fee amount (for home/EU) is highly regulated, what you get for it is not

38. Subsidies

39. Subsidy Expectations

40. Subsidy Expectations

41. Some questions • Should there be better regulation around what you get not just the fee? • How clear should agreements be on additional charges? • Should students know about detail of subsidies?

42. Costs preparedness • Students’ Unions indicated that it was necessary to investigate how much were students prepared for the costs associated with their degree. • Implications both for academic performance and non continuation • Scores highly in SU polls and SU Officer Manifesto analysis

43. Informed about costs? Worse for wp factors

44. What’s going on? I knew how much travel would cost, but the food at the university and cost of resources needed for my course are ridiculously high At school was always told ‘everyone can afford to go to university’ however if your parents, like mine, earn just over the cut off point for loans it is actually quite unaffordable. For example I live in the cheapest accomadation on campus and my loan is short nearly £2000 a year on rent, not including food and everything else, so actually I can’t really afford to be here. There isn’t any support or warning about how expensive it all really is, and unless your parents are on very low wages you don’t get any help whatsoever despite any other circumstances I looked online and made estimates of everything I needed before I enrolled to be sure that I would not run out of money at any time. nobody cares enough to explain to you that if you're middle class you don't get enough money from parents nor from the government so you're stuck in the middle with a small loan and a small amount of help from parents. Nobody told me watrose is more expensive than lidl I have been on many away days and widening participation schemes and they all told of the costs of university The only costs explained were those for tuition. The course had additional costs for printing and binding of theses which were not mentioned. No further information was offered about living costs, travel expenses etc. Did not take into consideration that professors would force us to buy their own books or we would 'fail' the course.

45. Costs Dissatisfaction related to • Lack of info re participation costs • Perception that costs levied by provider are rip off • Maintenance funding not covering costs Satisfaction related to • Clear information re direct costs • Social capital • Access to information re est participation costs

46. Some questions • What might be done to ensure students know TCP (Total Cost of Participation) • Are some provider levied costs monopolistic/anti competitive? • How can we encourage providers to consider impact on students of costs levied? • How can we encourage providers (and SUs) to intervene locally to reduce costs?

47. Outputs and Outcomes • Ongoing debate about whether “value” can or should be derived from the • quality of a provider’s services • its outputs or • its outcomes • Students’ Unions keen to find out which factors drove VFM perceptions

48. Outputs and Outcomes

49. Outputs and Outcomes NOTE: These don’t change for applicants or for graduates Comments suggest that students “get” that responsibility for quality factors is a provider’s, but outcomes is a shared responsibility

50. Some questions • Could TEF evolve to include more output measures (like QT proxies, A&F, learning resources) • Could TEF evolve to become more personalised (using big data for individuals) • Could providers be encouraged to convert baffling academic policies into clear (service) standards? • How can we drive efficiency within providers?