The Workplace Relevance Scale in the UES David Collings (ECU) and Bruce Guthrie (GCA )

1. The Workplace Relevance Scale in the UES David Collings (ECU) and Bruce Guthrie (GCA)

2. Workplace Relevance Scale • In this session: • Supplementing the UES • Why workplace relevance? • WRS Development • Source, versions, items

3. Supplementing the UES • Dennis Trewen (AO) past Australian Statistician “The university is key stakeholder. The success of the survey depends on their co-operation. This is more likely if they are significant users of the UES itself and also possibly use it as a vehicle to ask for information of particular interest to them. The Department should work on this possibly in collaboration with Universities Australia” Jan 2014

4. Supplementing the UES • SRC/GCA offering additional items and/or populations • 2012 UES – sector consultations with ACER, Melbourne CSHE etc • Internal focus • institution specific, localised concerns • Rationalising multiple surveys; • External focus • benchmarking opportunities • UES is inherently benchmarking oriented (funding device?) • E.g. Online Learning; Community Engagement; WIL • CEQ optional items as a model

5. ECU and workplace relevance • Below average employment outcomes (GUG! despite BGS) • Curriculum 2012 project – embed employability into curriculum • new ECU Strategic Priority – graduate outcomes • Teaching and Learning Functional Plan • 80% u/g courses with WIL and/or workplace elements • All u/g courses with employability activities • 80% of u/g courses with integrated careers and employability activities • 80% of u/g courses using e-portfolios • Careers unit broadened scope: • closer to faculties • Mentoring; Leadership; Volunteering

6. ECU and workplace relevance (cont) • Engagement (community includes industry/employers) • Course Consultative committees • WIL; products/services; guest lecturers etc. • Internal BGS phone survey Graduates (+12 mths and +24mths) • Rated a range of strategies re employability, e.g. in-class employer visits; jobsearch skills; career fairs. The two clearly most highly rated were: • “Courses and unis which include industry practicums or placements”: 63% of respondents rated 9/10 or 10/10 • “Courses and unis which have work-relevant content”: 61% of respondents rated 9/10 or 10/10 • Where to find a good measure of student perception rather than structural inputs or GDS outcomes ….with BENCHMARKING POTENTIAL

7. ECU and workplace relevance (cont) • AUSSE? – Work Integrated Learning Scale • ECU used this in AUQA 2012 • Some time-series • Some problems – • only 5 items, but 3 different response frames/scales ! • AUSSE supplanted by UES! • Griffith OLT study? • 2013 approx 12 unis; Calvin Smith • Over 100 items

8. BRUCE GUTHRIE • and • ALAN RICHARDSON to the rescue! • (via our esteemed ATN colleagues)

9. Workplace Relevance • Why workplace relevance? • Generic Skills (early 80s) • Graduate Attributes (last 10 years) • Higher Education Standards Panel (HESP) advises on the Threshold Standards of the Higher Education Standards for TEQSA

10. Workplace Relevance • Draft Standards for Learning Outcomes (Coursework) The learning outcomes for each course of study are informed by: • a) the mastery of specific disciplinary and/or interdisciplinary knowledge and skills that characterise the field of study • b) the generic skills and attributes required of graduates • c) the application of generic skills and attributes in the context of the field of study including the communication skills required, and • d) the requirements of employment related to the field of study

11. Workplace Relevance • ESS Pilot report gives a useful overview • Conceptual skills and attributes frameworks roughly split into • What graduates leave uni with • What their courses seek to impart • What employers need

12. Workplace Relevance Scale • Workplace Relevance Scale developed from open-ended CEQ responses arising from the development of the CEQuery package. • Alan Richardson & Boris Kabanoff • RMIT, UTS, QUT

13. WRS Original • The course developed an understanding of workplace skills. • The course provided the opportunity for putting theory into practice. • The course developed current professional skills. • The course was a good combination of theory and practice. • I got practical experience dealing with actual work situations. * • What I learnt benefited my future work. • The course helped prepare me for the workforce. • Subjects had no direct relation to the world of work (reverse scored) **dropped in subsequent version

14. WRS Original Updated • The course developed an understanding of workplace skills. • The course provided the opportunity for putting theory into practice. • The skills developed in the course were relevant and useful.* • The course developed current professional skills. • The course was a good combination of theory and practice. • What I learnt benefited my work. • The course helped prepare me for the workforce. *added in this version

15. WRS for the UES • WRS developed for graduates in the workplace • Past tense wording • Liaison with Alan Richardson, colleagues and GCA • Updated, shortened and fine-tuned for use in the UES

16. WRS for the UES • Five items, present-perfect tense: • Q9. The course has provided the opportunity for putting theory into practice. • Q12. The skills developed in the course are relevant and useful. • Q16. The course has developed current professional skills. • Q25. What I have learnt will benefit my future work. • Q29. The course has helped prepare me for the workforce.

17. WRS for the UES • Currently: • ECU • Swinburne • UTAS • UTS • ACU • Notre Dame • Open ended question for comments Any more?!

18. WRS for the UES • What value? • A scale is a better measure than single item • WRS has some decent validation behind it (at least for grads) • More likely to generate useful comments re employability • Benchmarking • Simple comparisons: where do we stand? • WRS comparisons less likely to be complicated by regional labour market factors than GDS outcomes, though discipline mix still a factor. • Who is doing well, and WHY/HOW? • Who does the work (data aggregation, liaison, reporting?) • CEQ optional scales? – how valuable have they been? (see national AGS reports for selection, but what of usage?) • Assess psychometric properties as UES add-on? (over to Alan?) • UES as appropriate vehicle for WRS vs GOS/ESS?