Nigel Ward University of Texas at El Paso

1. Fifth International Conference on Intelligent Technologies December 3, 2004 Dealing with Uncertainty in a Model of Computer Science Graduate Admissions • Nigel Ward • University of Texas at El Paso

2. (a 12 minute pre-banquet talk at a small 3-day gathering of soft-computing researchers)

3. ASolution enable applicants to predict acceptance decisions, using a web tool The Problem 10,000+ CS grad school applicants a year many wasted applications some disappointed applicants a model of applicant strength + models of admissions criteria

4. The Acceptance Estimator Concept demonstration

5. How to Combine GRE Scores? Two common styles: avg/sum and min: “we expect a GRE V+Q+A of at least 2100” “we expect at least 600 V, 700 Q and 650 A” A compromise: stronger scores weighted less, but not zero* 1.33 for weakest, 1.0 for middle, .67 for strongest (an ordered weighted averaging operator) * cf Carlsson, Fuller and Fuller in Yager and Kacprzyk, 1997

6. Sample Computation

7. department- specific threshold standard model of applicant strength x > GQ Explaining Apparent Diversity admissions policy for department x simplifications X’s published admissions policy and statistics spin fog guesses omissions

8. GPA JNTU GRE Composite Mumbai U. Texas at El Paso Estimating the Scaling Parameters To apply OWA, we must normalize scores first what is the GRE Q score corresponding to a 3.7 UTEP GPA?

9. Weighting the Scores ∑CL x IW i i CGRE = i ∑IW i i

10. Complexities in Recommendations • commeasurate with GREs and GPA • can be a plus or a minus • are fundamentally optional • are not expected to have specific points so no ranking factors • vary in influence so the importance weight computation is vital

11. Modeling Recommendations describing you as a Leading recommender is a = weight = scaling factor warmth score

12. NV = (RV - BV ) x SF i i i i CL = NV x RF i i i ∑CL x IW i i GQ = i ∑IW i i Summary of the Computation Subtract Baseline and Scale Raw to get Normalized Value: Order Normalized Values and apply Ranking Factors to get Contribution Levels: 2 r - 1 ( 1+ ) RF = 3 n - 1 i where r is rank, n is number of scores Weight and Sum:

13. Factors in Admissions Decisions In the Model • GREs • GPA • in-major or recent GPA • major • letters of recommendation • statement of purpose • scholarships • group membership • Not in the Model • undergrad school • GRE subject test (CS) • TOEFL • nationality/culture • specific coursework • research match • publications • etc.

14. compute GQ score > -25? Evaluation 55 UTEP applicant datafiles applicant features accept/reject decisions accept / reject 51/55 successes compare with failures explicable

15. published data for school X threshold for school X compute GQ score compute GQ score > accept / reject Modeling Other Departments applicant data

16. Does the Model Work for Departments?

17. Does the Model Work for Departments? Thus selectivity, as measured by the model, correlates with desirability, somewhat

18. The Diversity Behind the Numbers Minimum scores of 550, 600 and 3.5 on the verbal, quantitative, and analytical writing sections, respectively (U. of Delaware) Average scores of successful applicants to this program for Fall 2002: GRE: 560 verbal, 770 quantitative (U. of Houston) Most students admitted have earned scores in excess of 650 for the Analytical and Quantitative parts (Columbia)

19. inferred threshold Averages, Minimums, and Thresholds

20. inferred threshold Averages, Minimums, and Thresholds avg vs. threshold: ~20 (0.1 GPA points) ==> departments don’t have much variety (?) threshold vs. min: ~30 (0.15 GPA points) ==> departments don’t take risks (?)

21. A View of the Applicant Pool average minimum Number of Applicants acceptees Overall Applicant Strength (GQ score)

22. A Blurred View average minimum Number of Applicants acceptees Applicant Strength measured by GREs only

23. published data for school X threshold for school X compute GQ score compute GQ score > accept / reject Modeling Other Departments applicant data

24. threshold for school X compute GQ score compute GQ score adjustment description adjustment margin most above -30 40 average -20 20 > accept / reject hard minimums 10 40 soft minimums 10 30 Modeling Other Departments published data for school X applicant data

25. Presenting Uncertainty

26. Some Sources of Uncertainty • user interface errors • lack of information about the applicant • incorrect fundamental assumptions • incorrect GQ-model parameters • incorrect modeling of departments’ criteria • inadequate information on departments

27. Try it Yourself! http://www.cs.utep.edu/admissions/

28. Future Work • verification on data from more departments • better parameter estimates on more data • a more parameterized version to model different departments better • a centralized clearinghouse?

29. Benefits for UTEP • better informs potential UTEP applicants • increases site traffic, and applicant pool? • increases Google score • shows we understand student needs • makes the world a better place