M21 Assessment in the Workplace Biodata & The Interview Dr Caroline Bailey
Overview Biodata • Definition • Validity • Advantages & Disadvantages The Interview • Overview of the ‘typical’ interview • The quality of the interview as a selection technique : micro- and macro-analytic research • Structured Interviews : methods • Objective-Psychometric Perspective vs Subjective-Social Perspective
Overview • Psychometric ‘tests’: Cognitive Ability Tests & Personality Inventories • basic concepts • predictive validity • advantages & disadvantages • Assessment Centres : History of the AC technique; what is an Assessment Centre ? • Do AC’s work ? Predictive validity and the construct validity problem • Candidate reactions to AC’s • Future directions for the AC technique
A ‘typical’ interview Prevalence of use : Robertson & Makin (1986) : survey of major British employers identified in The Times 1000. Function : - various perspectives • the organisation • the applicant • HR / Occupational Psychology Moderators • the organisation = selection ratio • the applicant = the job market
Some variables in the interview (I) 1.Who does the interviewing ? • Stakeholders (HR, manager, panel) • Stage in the recruitment process • Information needed • Resources available 2.Structure of the Interview (Herriot) • Functions of utterances • Functions by parties • Content • Order
Variables in the Interview (II) 3. Content of the Interview • Taylor & Sneizek (1984) : UK vs USA graduate trainee selection interviews. • Keenan & Wedderbaun (1980) : consistency of topics across interviewers • Taylor & Sneizek (1984) : importance of topics with regard to overall selection recommendation. • reliance on biographical details
Variables in the Interview (III)Bias, Fairness and equal opportunities e.g. marital status, religion, home ownership, criminal record*, military discharge, pregnancy, spouse and family responsibilities ‘Questions posed during the interview (should) relate only to the requirements of the job. Where it is necessary to discuss the personal circumstances and their effect upon ability to do the job, this should be done in a neutral manner, equally applicable to all applicants’ Equal Opportunities Commission
How the interviewer reaches a decision • Olian, Schwab & Haberfield (1988) : meta-analysis : qualifications = 35% variance • Tucker & Rowe (1977) : decision in 9 mins • Springbett (1958) : • application form + appearance = predicted 88% of candidates’ ratings • weighting of information (+primary/recency effect) • Reliance on implicit personality theory • other stereotypes : age, physical attractiveness, clones, mirror images, contrast effects • Candidate’s impression management skills...
Quality of the Interview as a selection technique (I) • Wagner (1949) : 174 ratings, r = 0.23 - 0.97 (specific traits); r = -0.20 - 0.85 (overall ability). • intelligence (Sneddon, 1930) • sociability and likeability (Gifford, Ng and Wilkinson, 1985) • Weisner & Cronshaw (1988) : interviews based on formal job analysis
Quality of the ‘typical’ interview as a selection technique (II) Predictive Validity • Dunnette (1972) : 0.16 • Reilly & Chao (1982) : 0.19 • Hunter & Hunter (1984) : 0.14 Herriot (1991) cites • 0.14 (supervisor ratings) • 0.08 (promotion) • 0.10 (training success) • 0.03 (length of tenure) • overall : 0.14
Improving the Psychometric Quality of the Interview • Select the interviewers • Train interviewers • Tell interviewers what to look for (i.e. structured interviews) • Listen to candidates • Use the information efficiently
Definition of a structured interview ‘A series of job related questions with predetermined answers that are consistently applied across all interviews for a particular job.’ Pursell, Campion & Gaylord (1980) • Situational questions • Job knowledge questions • Job sample/simulation questions • Worker requirement questions
Types of Structured Interview (I) • Standardised Interview : Hovland & Wonderlic (1939) :covers work history, family history, social history and personal history. Valid predictor of those later dismissed. • Patterned Interview : McMurray (1947) : the interviewer has to rate aspects of how the candidate replied to a question. Intended to measure personality. • Patterned Behaviour Description Interview (PBDI) Janz (1982) • Comprehensive Structured Interview : Campion, Pursell & Brown (1988) • Structured Behavioural Interviews : Motowidlo et al (1992)
Types of Structured Interview (II) Situational Interview : Latham et al (1980) • Locke’s ‘Goal Setting Theory(1968) • [ Maas (1965) interview structure : traits] • Flanagan (1954) : critical incident technique • ‘future’ behaviours Validity studies • Hourly paid workers (v=0.46) • Foremen (v=0.30) • Entry level production workers : v= 0.39 (women) v = 0.33(Black)
Why do structured interviews have higher validity coefficients ? • based on thorough job analysis • assumes intentions and actual behaviours are strongly linked • same questions for each candidate • Wright et al (1989) ‘orally administered cognitive ability test’....
Perspectives on the interview Objectivist-Psychometric Perspective • US (particularly North America) • ‘non-participant observer’ • reduction in processual variance vs Subjectivist-Social Perspective • Schein, ’70,’78; Herriot,‘81,‘84,‘87,‘89 • ‘participating negotiatior’ • ‘processual negotiation’ • the psychological contract
Objective-Psychometric vs Subjective-Social Criticisms : Objective-Psychometric • the interviewer does play an active role • flexibility and adaptability of the interview • reliability : same info - interpreted differently • can interviewer dysfunctions ever be over-come ? Criticisms : Subjective-Social • candidate power ?
What is the best technique? Points to consider... • What needs to be assessed? Job analysis leading to person specification (KSA’s/Competencies) • Cost-benefit analysis of strengths/weaknesses of specific techniques (with regard to current selection situation) • Resources available : time, money, suitably qualified people others?..
The Assessment Centre Technique Originally used for military purposes in WWII, then Bray and Byham (1950’s)… first commercial AC (AT&T). 26 dimensions measured by : • a business game (‘the manufacturing problem’) • Leaderless group discussion (‘the promotion problem’) • In-Basket exercise (administrative exercise) • Measures of personality (incl. Thematic Apperception Test - TAT, and Incomplete Sentences Blank) • Interview
Prevalence and Nature of Use • Bray (1997) : 80% of Fortune 5000 companies use AC’s somewhere in the organisation • Shackleton (1991) : 21% of major UK org’s used AC’s in 1986; 59% by 1991 • Used in all industries (manufacturing most prevalent) • 1980’s = for selection of supervisors/middle management; 1990’s = for selection of ‘employees’ (non-mgmt) to executive
The AC Model (I) Guidelines : 17th International Congress on the Assessment Centre Method (1989) 1. Dimensions • Job analysis to identify Knowledge, Skills and Abilities (KSA) 2. Techniques • Must provide information for the dimensions identified in the JA. Multiple techniques must be used.
The AC Model (II) 3. Assessors • Must use multiple assessors to observe and evaluate each candidate. Assessors must be trained and demonstrate their ability to assess. 4. Gathering Data • Assessors must use a systematic procedure to record specific behaviour observations as they occur. Each assessor must prepare a report, and this data must be pooled to provide an OAR (overall assessment rating)
Other Variables in AC design • ‘Off the Shelf’ vs Tailoring • Length/Duration • AC model used [ Dimension-specific vs Task-specific ]
Candidate Reactions (face validity) • Dulewicz (1991); Thornton (1992) : candidate, assessors and management all hold positive attitudes toward AC’s : Candidates find exercises difficult and challenging - but believe measure job relevant behaviours and are fair. • Macan, Avedan, Pease & Smith (1994). AC vs tests : AC’s more acceptable, face valid and fair • NB. Candidate anxiety (Iles, Robertson and Rout, 1989) : 18%-32% of candidate said AC is stressful; Teel and DuBois, 1983 : 50% felt performance affected by stress.
Do AC’s work ? Criterion Validity (I) • Howard (1974) : criterion validity higher for AC’s than any other selection technique • Schmitt et al (1984) : AC mean validity = 0.41, equivalent to work samples, supervisor/peer ratings NB1. Developments in testing and interview techniques (and issue of incremental validity) NB2. Can cheaper measures substitute ? Lowry (1994) : AC and Personnel Records : AC strength = assessment of interpersonal skills.
Do AC’s work ? Criterion Validity (II) Gaugler et al (1987) : meta-analysis of AC validity : criteria = (1) career progress; (2) overall performance ratings (3) dimensional performance ratings (4) ratings of potential (5) wages (6) training performance Average ‘r’ = 0.40 (variance due to variation in components) Higher validities reported for AC’s with : (1) wider range of exercises (2) psychologists & managers as assessors (3) included peer ratings in OAR (4) have more female candidates.
The ‘Construct Validity Problem’ = although AC does show convergent validity, discriminant validity is usually poor. Why ? • The ‘classic’ explanation • criterion contamination • subtle criterion contamination • self-fulfilling prophecy • intelligence • consistency with past performance • assessors’ difficulty in providing ratings
Rating Techniques 1. Behaviour Checklists : • Reilly et al (1990) : for each exercise, assessors note ‘frequency of’ behaviours 2. Key Behaviours (Key Actions) : • Are models of effectiveness for handling situations. Capture quality and quantity of actions. Are not independent across dimensions.
Recommendations • Select fewer but more observable dimensions • Cover the important domains, but don’t expect sharp differentiations of dimensions that are subtle variations on the same theme (e.g. leadership) • Rate dimensions only AFTER have enough behavioural evidence to do so. • Define dimensions clearly and unambiguously
Candidate Reactions : Anxiety Fletcher, Lovatt & Baldry (1997) : state, trait and test anxiety to AC performance 38 candidates of an AC (8 dimensions, 7 ‘exercises’) Results • State anxiety had a curvilinear relationship to several AC measures, with low and high anxiety related to poor performance • Test anxiety sig -vely correlated to scores on a numerical test and a written exercise • High trait anxiety associated with better assessment ratings.
Future of the AC technique DeVries (1993) : critique of AC method (incl. AC model takes an outdated view of the manager) Bray and Associates (1992) : evi. of changes in dimensions assessed. • AC exercises - reflect empowerment of employees • Global market (cultural differences) • Technology (videotapes, computer sim’s, CAT, etc) • Used increasingly for selection of employees and executives • Used for a variety of other functions : RJP’s, placement, development, org. development, career and succession planning.
What is Biodata? (I) ‘autobiographical data which are objective or scorable items of information provided by an individual about previous experience (demographic, experiential, attitudinal) which can be presumed or demonstrated to be related to personality structure, personal adjustment or success in social, educational or occupational pursuits’ (Owen, 1976) Drakeley (1989) : classification of biodata : (i) background data (ii) commitment data (iii) achievement data
What is Biodata ? (II) Henry (1966) : dimensions of biodata include : • verifiable vs unverifiable, • historical vs futuristic • actual vs hypothetical • memory vs conjecture • factual vs interpretive • specific vs general • response vs response tendency • external events vs internal events, • strictly biographical vs attitudes
Use of Biodata • 1894 : Colonel Peters - Life Assurance company • Goldsmith (1922) : Weighted Application Blanks • 1960’s : introduction of Biographical Questionnaires • 1970’s : empirically derived biodata actual capacity of certain biodata to discriminate vs rationally derived biodata conceptual model - biodata that ought to be related to criterion
Biodata vs Personality • Biodata allow a definite, unique answer (personality inventories often don’t) • Personality inventories elicit ‘gut’ responses (biodata specifies in some detail exact information required) • Personality inventories - fixed key scoring vs biodata - re-keyed for each selection task.
Why should biodata be predictive of job performance ? • Psychological theories of personality that emphasize how developmental experiences shape our personality • One of the best predictors of future behaviour is past behaviour (hard biodata) • Goal setting theory - intentions predictive of behaviour (soft biodata) • WAB - invisible therefore IM more easily detected
Strengths of Biodata Strengths : Reilly & Chao (82) predictive validity can be as high as to consider biodata a legitimate alternative to standardised testing • systematic, uniform way to gather information • biodata questionnaires - multiple choice format which allows rapid processing of candidates • candidate reactions • theoretically, biodata are a more accurate representation of an individual than that obtained by (high IM) techniques such as the interview • minorities can be identified and treated fairly • biodata, once designed, are very cost effective
Weaknesses of Biodata • Homogeneity vs hetrogeneity • cloning the past • atheoretical • time consuming • content validity • biodata do not ‘travel • shrinkage over time