1. DLHE Good Practice London 17 September 2009

2. Objectives • Introduce good practice within DLHE record • Demonstrate tools designed to aid good practice • Provide institutional perspectives on completion of survey and data collection • Highlight data quality issues and checking

3. Target response rates

4. Target response rates • 80% for FT UK domiciled students (including Research Council students) • 70% for PT UK domiciled students • 50% all other EU

5. Difficulties in meeting targets • Increasing student mobility reduces the reliability of contact information • Public scepticism about what personal data is held against them at a governmental level • Survey saturation • High targets set

6. Current response rates • 78.5% for FT UK domiciled (-0.4% when compared with 2006/07) • 70.7% for PT UK domiciled (-0.4% when compared with 2006/07) • 54.7% for all other EU (+0.7% when compared with 2006/07)

7. Preparing for DLHE 2008/09

9. April target list system • Uses a student file from that year • Reduced validation • Generates an accurate population file to survey against • System currently underused

10. Population • Important to check the POPDLHE file generated from the Student Record submission… • …make sure you are not missing students… • …or that no one is going to be surveyed that shouldn’t be • The Match Report will show you how successful you were at this

11. Know your graduates • Up-to-date contact information • Some graduates will like to complete forms online, others will not, so offer as many methods of completing the survey as possible • Call at the right times • Identify your part-time and ITT graduates

12. Sections D and E • Ensure part-time and/or ITT students complete the sections relevant to them • Failure to do so will result in validation warnings • Be aware of students who changed to a PT mode of study at the end of their course but studied the majority as a full-time student

13. Telephone interview • Ask the respondent the core questions first if graduate is in a hurry… • …without them answered you do not have a valid response • Clarify contradictions or information that would appear incorrect

14. Location information • Location of employment information is very important to statutory users of data • Allows analysis of student’s geographical movement from entry to HE to the labour market • Try to code to full postcode, however at least outward postcode level or town postcode should be reported

15. Audit • The data should be kept only as long as is necessary • The completed questionnaires should be kept (as either a hardcopy or electronic version) for each individual leaver for one year after the data is returned to HESA • If the purpose is research then data may be kept for a longer period of time • There is no need to anonymise the data

16. Aardvark system changes • From now on, in addition to the Access Codes, users will also need a PIN code to create new accounts and/or add permissions to existing accounts. The PIN codes will be distributed by letter a few days in advance of the Access Codes • Both the Access Codes email and the PIN letters are sent to the nominated record contact at institutions • The system will now send an email to the transaction owner when a transaction has completed processing

17. DLHE online service

18. Why use it? • Help to improve response rates by offering an alternative to paper or phone based questionnaires • Students can complete it at their own accord and at their own pace • Intuitive and free to register and use • Built in validation

19. System Usage

20. Online DLHE Questionnairefor both January and April census Graduates complete form at www.dlhe.ac.uk or www.dlhe.ac.uk?INST=nnnn to bypass institution selection Institutions download data from submit.hesa.ac.uk

21. DLHE Online Statistics 1

22. DLHE Online Statistics 2

23. DLHE Online Statistics 3

24. Uses of online system • Existing institutions have used the system in different ways… • …either as an additional tool to the mailings and telephone follow-ups… • …or as part of a full integration policy • Many institutions also use the system as a data inputting tool for staff

25. Registering for system • Ahead of each DLHE survey you must pre-register for use of the system by the date we publish… • …and as such requires incorporation into planning

26. The Aardvark system • Register to use DLHE online on Aardvark • Control panel allows you to specify your requirements for system e.g. inclusion of institution's student number or including a PIN • Download the data of completed surveys from Aardvark

27. CASCOT

28. CASCOT (IER) • CASCOT is designed to assign a code to a piece of text. e.g. a SOC code to a job title from the DLHE survey • When CASCOT assigns a code to a piece of text it also calculates a score from 1 to 100 which represents the degree of certainty that the given code is correct • Available to download from: https://submit.hesa.ac.uk/aardvark/CASCOT.ASP

29. Benefits of using CASCOT • Ensures consistency within institution and also across sector • Reduces difficulty of interpreting text • Improves efficiency and reduces burden of data inputting

30. Potential problems • The quality of coding depends on the quality of the input text: "advertising & marketing" • These are the area of work. Are they an advertising executive, an advert designer or an advertising salesman? "answering phone enquiries“ - This is a job description, not a title.

31. Working with batches • CASCOT can be used to code on a record-by-record basis or can work in batches • Input files allow user to code as many records as they require • Setting up an output file will then store the results in a spreadsheet for you

32. Interpreting results • Always interpret the information accurately… • …however you do not have to use the recommendation made by CASCOT (regardless of the score it is given) • Use your judgement to assess whether it is the most accurate code and reflects the true nature of the job

33. For example • Student employed as an architectural model maker for which CASCOT gives the following code… • ‘Modeller (Architectural) 5315’ • …however that is classified as a non-graduate job… • …whereas ‘Modeller, Artistic 3411’ is a graduate job   

34. DLHE good practice The School of Pharmacy 17th September 2009 Mark Gittoes Head of Quantitative Analysis for Policy, HEFCE m.gittoes@hefce.ac.uk

35. Areas of use • Institutional/subject area reporting • Research • Understanding of provision • Overview of HE • Other bodies

36. Institutional/subject area reporting • HEI Performance indicators • Teaching quality information and UniStats • Regional profiles

37. HEI Performance indicators (1) • Proportion of graduates who are working or studying (or both) by institution • Currently published for full-time first degree graduates • Extending to other cohorts • Interest in the development of other indicators

38. HEI Performance indicators (2)

39. Unistats (1) • “Employment prospects” broken down by institution and subject area • Aimed at potential students • Reports: • Employment status • Top 10 profession types • Job category (graduate, non-graduate)

40. Unistats (2)

41. Research • Student ethnicity • Job types

42. Student ethnicity (1) • Relationship between a student’s ethnicity and experience in HE: • Entrant profiling • Progression through HE • Attainment in HE • Satisfaction • Graduate destinations

43. Student ethnicity (2)

44. Research: Job types (1) • Examination of Standard Occupational Classification (SOC) • Study of relationship with student perception of criteria needed for job? • Makes use of responses to DLHE question: • “Would you have been able to get the job you were doing on 14 April 2008/12 January 2009 without the qualification you recently obtained (the actual qualification, not the subject of study)?”

45. Research: Job types (2)

46. Understanding of provision • Strategically important and vulnerable subjects • Foundation degrees • Erasmus and placement courses • Further Education Initial Teaching Training

47. SIVS: Early careers of graduates “To address the lack of information on the supply and demand of STEM skills, HEFCE should … publish an annual report describing: undergraduate subject trends; recent graduate jobs and salaries; and the subjects where employers and government departments believe that there are, or are likely shortly to be, shortages of graduates with key skills.”

48. SIVS: Early careers of graduates

49. Foundation degrees: destinations Note: Full-time foundation degree qualifiers registered at English HEIs, 2005-06

50. Erasmus and placement courses:Salary six months after qualifying