WorkKeys Innovations: A Holistic Solution

1. WorkKeys Innovations: A Holistic Solution presented at the 2007 Michigan WorkKeys Conference Steve Robbins, AVP, Applied Research, ACT, Inc.

2. Overview • Why we should care about combining cognitive- and personality-based measures • WorkKeys Pyramid for Success • The National Career Readiness Certificate “Plus” and Personal Skills Assessments • Differential test strategies along the entire continuum of employment

3. What We Know from Research Literature on Incremental Test Validity • Cognitive and foundational skill tests “rule” • Adverse impact issues can be ameliorated with specific tests: • Task analysis • Job-specific tests • Combinational use of cognitive and non-cognitive tests • Personality and career tests add incremental validity

4. General vs. specific test effect sizes * Brown, Le, & Schmidt (2006) **Salgado et al. (2003)

5. Combining Personality & Cognitive Ability Tests • Level of correlations are low: GMA x C = .02 GMA x ES = .17 Math x C = -.15 Math x ES = .17 Reading x C = -.05 Reading x ES = .11 (Ackerman & Heggestad, 1997)

6. Personality Tests“Add Value” • Validity Estimates: 1 From Schmidt et al. (2007) using indirect range restriction

7. Combining Personality & Cognitive Ability Tests Creating opportunity for incremental validity especially as criteria vary Correlations between general cognitive ability and personality tests and measures of job performance in Project A Cog Pers Both Criteria .63 .26 .67 Core technical proficiency .65 .25 .70 General Soldiering Proficiency .31 .33 .44 Effort and Leadership .16 .32 .37 Personal Discipline .20 .37 .42 Physical fitness and military bearing (McHenry, Hough, Toquam, Hanson, & Ashworth,1990)

8. Why Now? • Market Need: SHRM National Study • Body of research informs how to optimize Personal Skill Assessments • ACT strength in cognitive & non-cognitive assessment (John Holland, VP Research in 1960’s) • Solution-focused approach

9. Businesses want integration of Cognitive & Personality Constructs SHRM: Applied Skills and Basic Knowledge: Combining and Ranking For new entrants with a two-year college/technical school diploma, applied skills are four of the top five “very important” skills in combined ranking with basic knowledge and skills. Casner-Lotto, J. & Barrington, L. (2006)

10. Solutions are Need Driven: Continuum of Employment

11. WorkKeys Assessment Solutions: Pyramid for Success • Ensure work and training readiness (WorkKeys Foundational Skills) • Give a “snap shot” of strengths and areas of improvement across key response tendencies or domains (“Talent”)

12. Pyramid for Success • FIT • Match individual interests/values to work environment • Enhance job persistence & satisfaction • Develop Talent pool to meet needs • TALENT • Benchmarking for selection • Coaching & Development • Compound Indices: Sales, Managerial, Leadership, Safety • PERFORMANCE • General Work Performance: Productivity, Absenteeism, Complaints about conduct • Safety and Risk Reduction • FOUNDATIONAL SKILLS • Job Analysis – identifies the skills and skill levels needed to be successful on the job • Assessments – show the current skill levels of an individual • Training – helps individuals and employers correct skill gaps

13. 6 5 Skill gap 4 Worker’s Skill Level 3 2 1 National Career Readiness Certificate The WorkKeys system assesses foundational skills, allowing individuals to seek a certificate, and when necessary to fill in the gaps. Silver Certificate Goal 4 4 4 3 3 Applied Math Reading Locating Information

14. National Career Readiness Certificate “Plus” • A flexible solution to meet state and system needs • Georgia example • Talent for coaching & development • Alterable variables allow for intervention & guidance

15. “It was about here, wasn’t it, Ed, when you came on board as sales manager?” Harvard Business Review. March 2007. p. 90

16. Talent Score Report

17. TALENT Development Worksheet

18. Behavioral Scales, Targets, and Representative Behaviors

19. Behavioral Scales, Targets, and Representative Behaviors

20. Behavioral Scales, Targets, and Representative Behaviors

21. Computer Programmers Sales Representatives & Managers A Comparison of Talent Scale Scores for Computer Programmers vs. Sales Representatives & Sales Managers 70 65 60 Mean Scores 55 50 45 Savvy Order Striving Stability Goodwill Influence Optimism Discipline Sociability Creativity Carefulness Cooperation Talent Scales

23. FIT Score Report

24. Fit Index 94 87 86 80 80 79 79 79 79 78 58 50 48 51 37 46 Employer Report 2 of 3 Fit Assessment Report for: Abbatoir Industries Site: Iowa City, IA Test Date: 3/30/07 Examinee: Alvin C. Tracey Examinee ID: XXXXX7890 Top 10 Occupations Ranked by Fit The top 10 occupations for the examinee, ranked by Fit Index, are shown below. This is based on all of the occupations in the WorkKeys Fit database. Examinee-specified occupations, if any, are in BOLD. Employer Report: Top 10 Occupations and Interest Inventory Results Code Title 41-3031.02 Sales Agents, Financial Services 13-2071.00 Loan Counselors Transportation, Storage, and Distribution Managers 11-3071.00 41-3021.00 Insurance Sales Agents 11-3011.00 Administrative Services Managers 41-9022.00 Real Estate Sales Agents 11-2022.00 Sales Managers 41-9021.00 Real Estate Brokers Middle School Teachers, Except Special & Vocational Educ 25-2022.00 13-2021.02 Appraisers, Real Estate Interest Results The Interest Inventory obtains scores on six scales. The examinee's score profile and highest scales are shown. Interest Score Profile Standard Score (20-80) 20 30 40 50 60 70 80 Administration & Sales: Persuading, influencing, or motivating others via sales, management, etc. Business Operations: Maintaining accurate/ orderly files/accounts via systematic procedures. Technical: Working with tools, instruments, machines, etc. Science & Technology: Studying phenomena through research, reading, etc. Arts: Expressing oneself via painting, singing, writing, etc. Social Service: Helping or serving others via teaching, counseling, etc. Highest Interest Scales Administration & Sales Science & Technology Business Operations

25. Employer Report 3 of 3 Fit Assessment Report for: Abbatoir Industries Site: Iowa City, IA Test Date: 3/30/07 Examinee: Alvin C. Tracey Examinee ID: XXXXX7890 Work Values Results Examinee's highest 5 and lowest 2 work values are shown. The definition of each value, as it appears in the Work Values Inventory, is provided. Employer Report: Work Values Inventory Results Highest Values Variety Using many different types of skills in my work. Public Contact Interacting with customers (as in sales) or the public (as in police work). Authority Telling people what to do; controlling the behavior of others. Precision Being exact or very accurate in the work I do. Helping People Improving the lives of others through activities such as teaching, physically assisting, or mentoring. Lowest Values Social Status Being looked up to by others in my company or my community because of my job. Physical Activity Moving around in my work by walking, bending, lifting, etc. For more information go to http://www.act.org/workkeys/assess/fit

26. Selection Solutions • Reducing Risk • Task Competence through WorkKeys and job profiling • General Work & Safety • Increasing Tenure • Task Competence through WorkKeys and job profiling • Fit • Getting the Right Person • Talent Benchmarking • Past Work Performance Record

27. “If they staged a slowdown, how would we know?” Harvard Business Review. March 2007. p. 90

28. Coaching & Development Solutions • Leadership Development • Talent • Fit • Teamwork • Talent

29. Return on Investment Approximations under Various Scenarios Notes: Selection % = the percentage of the candidate pool selected for hire, Candidate Success % = the percentage of the candidate pool that would be successful if hired, Selected Success % = the percentage of the selected candidate pool that will be successful, Cost per failure = the average cost for each unsuccessful employee relative to each successful employee, ROI per 100 candidates = the average return on investment for the selection procedure assuming a $15 fee per candidate.

30. Final Thoughts • Adverse Impact may be reduced when combining tests • Still recommend “multiple hurdles” approach • Incremental Validity Research underway: • Combination of Task & Personality measures • Differential work outcomes • ROI x Solution

31. References • Ackerman, P. L., & Heggestad, E. D. (1997). Intelligence, personality, and interests: Evidence for overlapping traits. Psychological Bulletin, 121, 219-245. • Brown, K. G., Le, H., & Schmidt, F. L. (2006). Specific aptitude theory revisited: Is there incremental validity for training performance? International Journal of Selection and Assessment, 14, 87-100. • Casner-Lotto, J. & Barrington, L. (2006). Are they really ready to Work? Society for Human Resource Management. http://www.shrm.org/hrresources/surveys_published • McHenry, J. J., Hough, L. M., Toquam, J. L., Hanson, M. A., & Ashworth, S. (1990). Project A validity results: The relationship between predictor and criterion domains. Personnel Psychology, 43, 335-354. • Robbins, S., Allen, J., Casillas, A., Peterson, C., & Le, H. (2006). Unraveling the differential effects of motivational and skills, social, and self-management measures from traditional predictors of college outcomes. Journal of Educational Psychology, 98, 598-616. • Rotundo, M., & Sackett, P. R. (2002). The relative importance of task, citizenship, and counterproductive performance to global ratings of job performance: A policy-capturing approach. Journal of Applied Psychology, 87, 66-80. • Salgado, J. F., Anderson, N., Moscoso, S., Bertua, C., & de Fruyt, F. (2003). International validity generalization of GMA and cognitive abilities: A European community meta-analysis. Personnel Psychology, 56, 573-605. • Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262-274. • Schmidt, F. L., Shaffer, J., & Oh. I. (2007). Reassessing the Relative Importance of Cognitive Ability and Personality in Job Performance and Training Performance: Some Surprising New Research Findings. Paper presented at the 2007 ATP conference, Palm Springs, CA. Feb. 6.

32. Incorporating Foundational and Soft Skill Assessments For questions regarding this presentation or for further information contact: Steve Robbins at 319-337-1227 or steve.robbins@act.org, Gary Nolan at 319-337-1526 or gary.nolan@act.org,