Presented by Dr Clare Hourigan University Planning & Statistics, Office of Planning & Quality

1. Presented by Dr Clare Hourigan University Planning & Statistics, Office of Planning & Quality The use of the general achievement test in student selection at Monash University: A preliminary evaluation

2. The Current Higher Ed Environment • 2008 Review of Higher Education • Increase bachelor qualification attainment (40% of 25-34 year olds by 2025) • Increase participation of low SES students (20% of undergrad population by 2020). • TEQSA • New round of quality audits are likely to focus on student performance

3. Role of Admission Policies • Admission policies will play important role in meeting these challenges. • Aim is to grow and diversify without compromising on quality. • Important to continue to select students that are likely to succeed • Must aim for a student population diverse in social background but not academic ability.

4. Use of Year 12 results in selection • Year 12 results (ENTER in Vic) current primary tool used for undergrad student selection. • Best predictor of university performance that we have but there is room for improvement. • Monash internal analysis shows it can account for 35.4% of variance in first year marks but varies by course/field of education (Hyndman & Akram, 2006)

5. Use of Year 12 results in selection • Reliance on Year 12 results may be part of the reason why low SES students are under-represented at university. • Year 12 completion rates lower (James, Anderson, Bexley, Devlin, Garnett, Marginson & Maxwell, 2008) • and ENTER scores are lower among low SES students (Birrell, Rapson, Dobson, Edwards & Smith, 2002)

6. Use of Year 12 results in selection • Why do Low SES students obtain lower year 12 results? • Aspirations/encouragement from peers/family/teachers (James, 2002) • School resources (Edwards, Birrell & Smith, 2005)

7. Use of Year 12 results in selection • Less prepared for university or do their marks under-reflect their ability/potential? • Study of Monash students found, when ENTER was controlled for, students from non-selective Govt schools performed best (Dobson & Skuja, 2002) • Challenge for universities is to compensate for disadvantage to the extent that it has resulted in Year 12 results that under-reflect likelihood of success at uni.

8. Current policies to compensate for disadvantage • Special Entry Access Scheme in Vic • Bonus points given to applicants for different equity categories including socio-economic disadvantage. • Systematic approach but qualitative in judgement of extent to which Year 12 results have been affected.

9. Current policies to compensate for disadvantage • Under-represented schools • Assumes all students from particular schools were disadvantaged in the same way.

10. Aptitude Tests • 2004 Admissions to Higher Education Review UK • Recommended aptitude tests be considered by universities in student selection to ‘help uncover hidden talent’.

11. Aptitude Tests • UK recommendation based on pilot of Scholastic Assessment Test (SAT). • A levels and SAT both influenced by SES (high SES students do better) but measure different aspects of academic ability because some students do well on one but not the other. • Students from schools with lower A levels least likely to score well on both tests (21% compared to 44%). • If uni selection used both A levels and SAT results, uni offer rates for students from low attaining schools would double.

12. Aptitude Tests • Monash started considering using admission tests to enhance selection. • In 2007 trialed the UniTest at Berwick Campus but can be administratively costly and places extra burden on students. • GAT examined because it’s undertaken by all VCE students

13. General Achievement Test • Monash undertook analysis into predictive validity of the GAT on 1st year marks. • ENTER is best predictor (35.4% of the variance) • GAT also strong predictor (32.3% of the variance) • Using both GAT and ENTER can improve predictability by 3% and predictive nature of GAT is stronger at lower ENTER ranges.

14. Use of GAT in selection at Monash • In 2008, Monash commenced 3 year pilot to use the GAT in middle band selection of VCE applicants with the aim to: • Differentiate between middle band applicants to select those most likely to succeed • Identify and select students likely to succeed at university but had underperformed in year 12 due to personal circumstances or educational disadvantage

15. Use of GAT in selection at Monash • In 2009, GAT predicted ENTERS (GPENTERS) were used in middle band selection. • Middle Band: Approx 20% of places set aside for assessment on range of extra criteria • Performance in selected VCE subjects • SEAS bonuses • Bonus aggregate points (2 to 9 points) where applicants GPENTER > ENTER

16. GAT Preliminary Evaluation • Based on students who commenced in 2009 as selection method changed. • Did use of the GAT assist in: • selecting applicants likely to succeed who otherwise would not have been selected? • diversifying Monash’s commencing student cohort with regards to socio-economic status?

17. Performance of GAT offer students First semester average marks and progress rates among domestic VCE school leaver students commencing a bachelor’s pass degree at Monash University by offer type, 2009

18. Can GAT mediate for effect of SES on ENTER? • Examined relationship between SES and ENTER and SES and GPENTER

21. Can GAT mediate for effect of SES on ENTER? • Perhaps differences in ENTER/GPENTER due to true differences in potential to succeed. • If true then uni performance should not differ by SES status when ENTER/GPENTER is controlled for. • Used linear regression model to examine relationship between ENTER/GPENTER and SES status and uni performance (first semester marks).

22. Can GAT mediate for effect of SES on ENTER? • R Square = 0.21, Standard Error = 11.32 • Extent to which year 12 results under-reflect uni performance similar for low and medium SES students • Perhaps Year 12 results of high SES students are inflated

23. Can GAT mediate for effect of SES on ENTER? • When GPENTER used: variation across SES status was not significant. • GPENTER was not particularly strong predictor of unipeformance but less likely to under-reflect likelihood of success of low and medium SES students • R Square = 0.08, Standard Error = 12.18

24. Did the GAT help Monash select more low SES students? • 20,830 VCE school leavers applied to Monash in 2009 • Examined eligibility for GAT bonus • Middle band offer rates by SES status

26. Did the GAT help Monash select more low SES students? • Overall, low (and medium) SES students more likely to be qualify for a GAT bonus • However, many of the low SES applicants did not meet the eligibility score ruling. • Among applicants above eligibility score, qualification for GAT bonus almost same for low and high SES applicants

28. Did the GAT help Monash select more low SES students? • GAT offer rates among middle band applicants were: • Low: 2% • Medium: 2.3% • High: 1.6% • GAT did not assist Monash in diversifying the student population with regards to low SES status predominately due to eligibility score ruling.

29. Conclusion • Findings are preliminary and we will be monitoring this as the pilot progresses. • Results are promising • 163 students who would not otherwise have been selected were given an offer • These students performed as well as other middle band students.

30. Conclusion • The use of the GAT did not help diversify Monash’s student population with regards to SES. • This was predominantly due to the eligibility score ruling. Many of the low SES applicants did not meet the eligibility score. • Results may differ for universities with wider range of eligibility scores