substantive assessment: abilities and personality

1. Substantive Assessment:Abilities and Personality Prof. John Kammeyer-Mueller MGT 6366

3. Remember—do you want to hire for the person or the job? General characteristics Examples: Intelligent Hard working Gets along with other people These are good for all jobs (but maybe more important for some jobs than others) Job specific characteristics Examples: Knows what the job involves Has skill at doing the things that are required These are good for some specific jobs 1 Huffcutt, A. I., & Arthur, W., Jr. (1994). Hunter and Hunter (1984) revisited: Interview validity for entry level jobs. Journal of Applied Psychology, 79, 184-190.1 Huffcutt, A. I., & Arthur, W., Jr. (1994). Hunter and Hunter (1984) revisited: Interview validity for entry level jobs. Journal of Applied Psychology, 79, 184-190.

4. Hiring for the Person:Traits for Selection Identifies abilities and other characteristics for a job Usually traits can be measured using professionally developed tests with high reliability Addresses cases where applicants are unlikely to have the skills you want them to develop Traits must be stable over time Arising from genetics or childhood experiences Arising from learning Emotional states or moods are too transitory

5. Stability of Core Affect and Personality Scores Evidence shows considerable test-retest correlations of personality scores over period of 2 ½ years Note that stability tended to be lower for affect scales Vaidya, Gray, Haig, & Watson, 2002

6. Longitudinal Stability of Personality as a Factor of Time

7. Persistent Effects of Personality Evidence shows that personality traits measured in 10 year olds can predict behavioral outcomes twenty years later Shiner, Masten, & Roberts, 2003

8. Stability of Various Measures of Personality and Other Life Outcomes

9. Behavioral Genetics: Human Building Blocks Simple genetics Chromosomes Genes The biggest genetic difference Gender (X vs. Y chromosome) Chromosomal abnormalities strongly linked to criminal behavior

10. A Few Observed Heritability Coefficients (MZA studies)

11. Behavioral Genetics and Individual Differences Important caveats Genes are not deterministic for psychological traits Most studies conducted in a limited range of environments; more extreme environments would show lower heritability Controversy: Can companies perform genetic screening?

12. Measurement and Meaning of Abilities Definition and parameters of GMA Measurement techniques Correlations with multiple dimensions of job performance The problem of disparate impact Other predictors

13. General mental ability for selection Critical area for measurement Everyone agrees that they want smarter employees Intelligence would seem to matter for every single aspect of job performance There is evidence that this is something that is fairly stable within a person Critical area of controversy Genetic inheritance and the “cognitive elite” Adverse impact statistics

14. General mental ability for selection What is it? It goes by many names: g, general mental ability, IQ, intelligence A general measure of cognitive functioning that should work across several different domains First proposed by Francis Galton, an English geneticist and relative of Darwin It remains one of the most studied of all human characteristics

15. General mental ability for selection After taking into account gender and physical stature, brain size is correlated about .40 with IQ The speed of nerve conduction is also correlated with IQ Energy expended during problem solving is inversely related to IQ levels The brain waves of individuals with higher IQs respond more quickly to simple sensory stimuli (clicks, lights)

16. A Conceptual Depiction of Mental Abilities

17. Just how positively correlated are these subdimensions?

18. General mental ability for selection So what’s the alternative? The SAT/GRE dimensions Howard Gardner’s theory of multiple intelligences How does data support multiple intelligence? Using specific abilities provides no increase in predictive power Using any subset of specific abilities is typically worse than if just a single dimension was used

19. Correlates of general mental ability Childhood outcomes Average r=0.50 with school performance However, children in Japan and China know more math than American children even though IQ scores are very similar Average r=-0.19 with juvenile crime General psychomotor skills Reaction speed r=.30 Judgment speed r=.55

20. General mental ability and stability Is it really a trait? Is it stable? Test re-test Age 6 to 18=0.77 Age 12 to age 18=0.89 Even higher correlations if tests are given on multiple occasions What does this mean? It means that rank order is very stable over time It does not mean that an average (100 IQ) 6 year old can do what an average 18 year old can do

21. General mental ability and stability The genetic inheritance of GMA Two statistics: h2 (genetic effect) c2 (shared effect) The effects of heredity only get stronger with age

22. However, a rising tide can raise all boats Most test scores show the Flynn effect Scores on IQ tests have been steadily increasing over time Although tests are consistently updated so each test has an average score of 100, evidence shows a clear change of about 3 points per decade The gains are largest among Latinos and African-Americans

23. Examples of items from the Wonderlic Personnel Test Fairness in Employment Testing: Validity Generalization, Minority Issues, and the General Aptitude Test Battery (1989) Fairness in Employment Testing: Validity Generalization, Minority Issues, and the General Aptitude Test Battery (1989)

24. Examples of items from a government ability test (GATB)

25. Examples of items from a government ability test (GATB)

26. Data clearly show that general mental ability as measured by the ASVAB is correlated with education levels, income, self-esteem, and weeks of unemployment even with a 10-year gap between measures 1 Spearman, C. (1927). The Abilities of Man. New York: Macmillan. 2 Neisser et al., (1996). Intelligence: Knowns and unknowns. American Psychologist, 51, 77-101. 3Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262-274. 4 Hunter, J. E. (1980). Validity generalization for 12,000 jobs: An application of synthetic validity and validity generalization to the General Aptitude Test Battery (GATB). Washington, D.C.: U.S. Department1 Spearman, C. (1927). The Abilities of Man. New York: Macmillan. 2 Neisser et al., (1996). Intelligence: Knowns and unknowns. American Psychologist, 51, 77-101. 3Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262-274. 4 Hunter, J. E. (1980). Validity generalization for 12,000 jobs: An application of synthetic validity and validity generalization to the General Aptitude Test Battery (GATB). Washington, D.C.: U.S. Department

27. GMA for selection 1 Spearman, C. (1927). The Abilities of Man. New York: Macmillan. 2 Neisser et al., (1996). Intelligence: Knowns and unknowns. American Psychologist, 51, 77-101. 3Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262-274. 4 Hunter, J. E. (1980). Validity generalization for 12,000 jobs: An application of synthetic validity and validity generalization to the General Aptitude Test Battery (GATB). Washington, D.C.: U.S. Department1 Spearman, C. (1927). The Abilities of Man. New York: Macmillan. 2 Neisser et al., (1996). Intelligence: Knowns and unknowns. American Psychologist, 51, 77-101. 3Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262-274. 4 Hunter, J. E. (1980). Validity generalization for 12,000 jobs: An application of synthetic validity and validity generalization to the General Aptitude Test Battery (GATB). Washington, D.C.: U.S. Department

28. Overall it seems like GMA is a perfect selection tool It’s very stable over time It’s related to measures of learning ability and adaptability It’s one of the best predictors of job performance available The tests are cheap (about $2.00 per applicant tested) It really doesn’t matter which test you use since they all measure the same thing But…

29. …there are potential problems with GMA test scores The subtests show differences for gender Women score higher on verbal intelligence Men score higher on visual-spatial intelligence and the math section of the SAT These differences are not very great and there are massive overlaps However, overall IQ scores are basically equivalent for men and women

30. GMA Scores and Race/Ethnicity There are clear differences in GMA scores based on ethnicity African-Americans scored about 1 SD below Whites, although more recent data suggests this difference is closer to 0.6 to 0.7 SD Latinos score between Whites and African-Americans Asians score slightly higher than Whites Ashkenazi Jews score about ½ SD higher than other White groups

31. GMA Scores and Race/Ethnicity Linguistic bias? efforts to remove cultural loading from questions have a fairly minor effect on scores using pictorial tests does not necessarily alleviate differences Stereotype threat? African American test scores are lowered if they are told that tests are measuring mental ability There is not a similar drop in test scores among Whites These results do not always generalize, but they do suggest we should consider motivational issues Presentation format? Chan and Schmitt (1997) and Pulakos and Schmitt (1996) found much lower group differences in tests presented verbally, although this is a complex relationship which is not found in all verbal tests Verbal tests seem to add additional constructs

32. Returning to our conception of biased from earlier GMA tests predict just as well for both groups, but one group gets lower scores This passes the equity standard, but not equality There is no factual or logically rigorous reason to say this is okay or not

33. Bobko, Roth, & Potosky:Cognitive Ability and Alternative Predictors

34. Bobko, Roth, & Potosky:Cognitive Ability and Alternative Predictors Using alternative predictors (composite of all three) does a lot to increase AI ratios Any use of cognitive ability causes problems

35. Bobko, Roth, & Potosky:Cognitive Ability and Alternative Predictors Organizations need to use “alternative predictors” with the lowest possible adverse impact Multiple correlations and d scores: Four predictor multiple R=0.43; four predictor d = 0.76 Alternative predictor multiple R=0.38; alternative predictor d = 0.36

36. Applicant reactions to GMA tests In general, results suggest applicants tend to find GMA tests acceptable They especially like tests that are called “personnel tests” They like tests more when they are paired with individual interview information They usually like tests of knowledge and abilities more than they like personality tests or biodata.

37. The importance of social skill in GMA/performance relationships This graph shows that social skill is positively related to task performance However, the best performance comes from people with high social skill and high general mental ability Those with high GMA do no better than those with low GMA when their social skill levels are equivalent

38. Measurement and Meaning of Personality Levels of measurement Conscientiousness Extroversion Agreeableness Openness Core self-evaluations Neuroticism

39. The personality conundrum We can train people to do things where skills are concerned. But there is one capability we do not have and that is to change a person’s attitude. So we prefer an unskilled person with a good attitude…to a highly skilled person with a bad attitude Herb Kelleher, CEO, Southwest Airlines Most organizations want to hire people based on their personalities, but personality is notoriously difficult to measure

40. Personality and Work:Conscientiousness This is the tendency towards orderliness, dutifulness, achievement striving, self-discipline, and caution What are the positive behaviors associated with this trait? Are there any potential downfalls? Which jobs would be most likely to demand high conscientiousness?

41. Personality and Work:Conscientiousness Summary of processes Increases goal setting behaviors Increases self-efficacy Increases value placed on social order and conformity Summary of situational effects Stronger effects when situations are weak or when supervision is non-existent Average conscientiousness of a group is related to group performance Can actually decrease performance for novel tasks More likely to be entrepreneurs

42. Personality and Work:Extroversion This is a tendency towards friendliness, gregariousness, assertiveness, activity, and excitement seeking What are the positive behaviors associated with this trait? Are there any potential downfalls? Which jobs would be most likely to demand high extroversion?

43. Personality and Work:Extroversion Summary of processes Decreases blood flow to frontal lobes Both introverts and extraverts are trying to regulate their level of psychological arousal Associated with increased self-efficacy Linked to achievement motivation Summary of situational effects More important in social situations like leadership Can increase citizenship behavior (helping) in some social situations Extraversion is a hindrance in distributive bargaining Higher variability in agreeableness linked to superior group performance

44. Personality and Work:Agreeableness This is a tendency towards trust, morality, altruism, cooperation, modesty, and sympathy What are the positive behaviors associated with this trait? Are there any potential downfalls? Which jobs would be most likely to demand high agreeableness?

45. Personality and Work:Agreeableness Summary of processes Associated with values of benevolence and traditionalism Has a relatively large negative relationship with goal-setting Breaks into two dimensions—morality and conflict avoidance Summary of situational effects Agreeable individuals are more helpful in minimally constrainted situations Agreeable individuals prefer tasks calling for helping, but dislike tasks calling for conflict Agreeableness does not moderate helping friends or family—but agreeableness does moderate helping strangers Agreeableness is related to citizenship and helping performance in groups (not surprisingly) Agreeableness is a hindrance in distributive bargaining

46. Personality and Work:Openness to experience This is a tendency towards imagination, artistic interests, emotionality, adventurousness, “intellect”, and liberalism What are the positive behaviors associated with this trait? Are there any potential downfalls? Which jobs would be most likely to demand high openness?

47. Personality and Work:Openness to experience Summary of processes Increased activity in the dopamine systems More flexible organization of ideas Mildly linked to goal setting motivation Place more value on universalism and self-direction; tend to distain conformity and tradition Summary of situational effects Open individuals learn faster in situations calling for change Openness is very strongly linked to creativity More likely to be entrepreneurs

48. Hurtz and Donovan: Meta-Analyzing the Big Five We find conscientiousness is the best predictor across all jobs The validity of the other dimensions is fairly poor

49. Hurtz and Donovan: Performance Domain May be a Moderator

50. Dimensions of personality: Meta-analytic results These studies by Judge and Bono, and Judge, Bono, and Locke show that personality is related to satisfaction and leadership. These relationships are generally stronger than the relationships with task performance.

51. Core Self-Evaluations Scale items I am confident I get the success I deserve in life. I am capable of coping with most of my problems. There are times when things look pretty bleak and hopeless to me (R). When I try, I generally succeed. I determine what will happen in my life. I am filled with doubts about my competence (R)

52. Core Self-Evaluations These are measures of core self-evaluations Typical features Positive self image (self-esteem and self-efficacy) Internal locus of control Low neuroticism Stability can be inferred from self-esteem measures Test-retest correlations among adults over periods around two years typically average around 0.60 Lower stability in very young and very old individuals

53. Traits that combine other good predictors Why do these traits hang together? Locus of control and self-image Self-image and emotional stability Locus of control and emotional stability Why would self-evaluations matter for work related outcomes? Job satisfaction Job performance (r=.26)

54. CSE and Satisfaction:Path Model Positive people describe their jobs as having good characteristics This makes them more satisfied Their job satisfaction spills over to their lives in general

55. CSE and Performance:Path Model People with positive self-evaluations set higher goals for themselves, which is a major reason for their higher levels of performance

56. The Importance of Situational Strength Evidence from Colbert, Mount, Harter, Witt, & Barrick, 2004 Conscientious people work equally hard regardless of work environment; people low in conscientiousness withhold effort in a negative work environment Agreeable people are positive towards co-workers regardless of work environment, people low in agreeableness are more negative towards coworkers in negative work environments Stress and coping literature In moderately stressful situations, neuroticism is strongly (negatively) linked to job satisfaction and strain, in very high stress situations, personality is less significant

57. The Importance of Situational Stability Research on changing task environments: flight crew studies Individuals high in openness to experience learn to adapt their behavior to new situations more rapidly Research on organizational socialization Individuals high in extraversion and openness to experience take more steps to fit into new work environments, and adapt more successfully

58. Examples of psychopathology items Neuroticism1 I dislike myself.  I am often down in the dumps.  I have frequent mood swings.  I panic easily. Items that are sort of like MMPI items (note that access to the MMPI is very closely legally regulated) My teeth itch The color red appeals to me I hear voices telling me what to do

59. Personality and Work:Neuroticism Summary of processes Increased sensitivity to environmental stimuli due to activation of the sympathetic nervous system Linked to worry, negative emotional states, and increased use of avoidance coping strategies Negatively linked to all aspects of motivation Summary of situational effects Less likely to do well as entrepreneurs Higher variability in neuroticism negatively linked to team performance Interesting—not more likely to turnover from jobs and relationships with performance as a whole are weak

60. Examples of psychopathology items The problem is with the ADA Employers cannot ask questions related to disabilities prior to making a job offer. An employer can conduct such an investigation after employment as part of a routine medical examination. Employers can refuse to hire, discipline, or discharge an individual for conduct caused by a disability, but they can’t act because of the disability itself. Mental illness is common (13-20% of individuals experience some form of depression in their lives, 5.1% experience anxiety disorders, about 1% experience schizophrenia) Most neuroticism measures capture something related to mental illness, but they are not measures of mental illness per se.

61. Examples of psychopathology items How to proceed if mental health is important Begin by testing the current population of job incumbents for mental health Compare high vs. low scorers in terms of job performance (validation study) Also look for archival data to compare your job incumbents to the population at large Demonstrate a substantial risk to the population at large if individuals with a specific psychological disability are able to work in your company Take steps to provide a more psychologically healthy work environment (lower stress)

62. Specific Subtraits: The Importance of Integrity Integrity or honesty tests Traditional justification for use Reduce employee “shrinkage” Reduce white-collar crime in some professions Typical features of high integrity Honesty, morality, and ethical behavior High levels of conscientiousness (dutiful) High levels of agreeableness (moral, trusting) Low neuroticism (less anger and immoderation)

63. Getting At Integrity Overt use Did you ever make a false insurance claim for personal gain? Have you ever stolen anything? Do you believe that you are a trustworthy person? Covert use What percentage of people take more than $5 per week (in cash or supplies) from their employer? What proportion of workers do you think have falsified their timecards?

64. Traits that combine other good predictors Relationships with important constructs Related to general job performance r=0.34 Related to lower counterproductivity r=.47 This includes things like absence, theft, and violence It does not seem to matter if the measure is overt or covert Another positive result: integrity tests do not have a demonstrated adverse impact against any demographic groups

65. Given all this, where does our “Chicken Little” scale fall? Remember the items Do you wake up several times in the night to check on your child? Do you feel "doom" is lurking around every corner? Do you always visualize the worst-case scenario? Does your worrying irritate those around you? Do you feel like there is always a crisis to deal with? Do you think this is a measure of something that already exists?