Unit 2: Performance Assessment & Measurement

Unit 2: Performance Assessment & Measurement Traditional Performance Appraisal Performance Assessment Performance Measurement

Unit 2 • Unit exam over study objectives (27 points) one week from tonight • Muchinsky: traditional performance appraisal • Daniels chapter: behavioral measurement & performance matrix • Pampino et al. article - great example of use of performance assessment, multiple component intervention, and social validity assessment. Only problem is short-term nature of the study • Iwata et al. article - antecedents vs. antecedents + consequences • Pinpointing exercise (8 points) due Wed • Description of performance measurement project

Pinpointing exercise • For a job you have had or are familiar with, identify at least 3-4 key performance measures (behaviors or products of behavior) • Measures should be objective that could be targeted in a PM intervention • If possible, include 1-2 measures that could be improved and indicate “improvement” after those • Format • Typed • Put job title/position at the top • Just bullet your measures unless you feel they need to be explained

Performance Measurement Project • Proposal due Monday, Oct. 15 (35 pts) • Final project due Monday, Dec. 10 (35 pts) • See syllabus for policy on lateness • Two options • Development and implement a measurement system for a minimum of 4 weeks • If measures already exist and you can obtain those measures from archive for 4 weeks, implement an intervention (Final project due Monday, 12/3 if grade before ME2)

Project description • You may work with in groups or by yourself • If you work in a group, each individual will receive the same grade - can’t separate individual contributions • Job • One or more workers: one is OK • Full-time or part-time • A friend, relative, significant other, another student, someone you supervise: only restriction - not one for yourself • Business or educational setting (i.e., PSY 3600 lab instructors or teachers, PSY 1000 assistants, servers) • Advise against secretaries and programmers

Some basic rules related to employment jeopardy • The emphasis will be positive - I will not accept a project in which the objective is to document poor performance • You must inform the worker or workers whose performance will be measured; if possible, give employees an option • If the supervisor or manager will see the data, you must insure that he/she will respond to it positively, not negatively • If supervisor responds negatively and you and I cannot resolve that, then I will have you terminate the project

HSIRB issues • Research: a systematic investigation that contributes to generalizable knowledge • Projects in this class do not constitute research, however, they could (more later) • HSIRB class registration of projects • I have registered the class with the HSIRB and gotten initial approval for the projects • After the proposal, but before you start the project I will send titles to HSIRB for final approval • You cannot implement your project until I get this final approval!!

Why aren’t the projects research? • The data will only be used in-class or internally by the organization: thus do not contribute to generalizable knowledge • If you use this as a pilot for your thesis or present the data publicly, then the project becomes research: Why else would you do that if the data don’t contribute to generalizable knowledge? • Confidentiality issues arise when projects are publicly presented: cannot violate a participant’s right to confidentiality You must submit a full HSIRB protocol if you use this as a pilot study for your thesis or present it publicly!!! (and before you can do that, on-line training; sample of a full protocol in course pack)

Why aren’t the projects research? (cont.) • In OBM, you are being invited by management to implement a management system • Organizations have the right to implement and evaluate management systems as part of their normal business practice • They can require all employees to participate in that system so you do not have to get consent from the employees (you do need a letter of consent from management) • You are not a researcher who is implementing the intervention, you are implementing or evaluating the system on behalf of management (consultant, agent) • With the caveat that data will only be used internally (important issues, related to projects, theses, questions?)

Ethical questions to ask yourself • Can the employees be harmed by the data you collect - are you placing them in employment jeopardy? • Group vs. individual data? (Ok to use individual data) • If individual data, can the individuals be identified? • Are you using the least restrictive and least intrusive procedure that is effective? (aversiveness) • Publicly posted individual data with identifiers vs. group data or individual data that are coded • Confidentiality of the employees • Should not discuss performance of individuals with identifying information with anyone outside the organization • Proposal and final report to me: do NOT include the name of the organization or employees

Performance Measures • Objective • System may cover: • 2-3 measures that need to be improved • Measures that target only the critical performances • Measures that cover all of the major responsibilities • Must be repetitive - daily or weekly measures • Verifiable - if you ask the worker to self-record you must develop a system to verify accuracy • Strongly recommend accomplishments rather than behaviors • Behaviors require you to be observe the person while working - very labor and time intensive (biggest problem in past) (in syllabus, mention a book by Abernathy provides measures)

Measurement system and forms • What will be measured and why the measure is important • Who will measure? • When will measurement take place? • How frequently will measures be taken? • If self-report, how will the data be verified? • Include the measurement form • Sample graph or data summary - how are you going to present the data?

Proposal format and grading • Both format and grading are discussed in the project description • A sample proposal and sample final project reports are included in the course pack in this unit • Strongly recommend you get started now - proposals are due in 3 1/2 weeks • Yes, I will take them early, if you want to get started early QUESTIONS??

Traditional Performance Appraisal: Muchinsky • SO1: Characteristics of performance appraisals that affect the way the courts rule • Performance appraisals are subject to federal and state EEO laws, and more and more cases are being filed. • Protected classes under Title VII? African Americans, Hispanics, Asians, Native Americans, and females. (on click)

SO1 Five characteristics that affect the way courts rule • Should be objective (NFE, no brainer) • Should be job related, preferably based on a written job analysis (NFE, no brainer) • Should be based on behaviors, not traits • Research shows no difference between trait-based forms and behavior-based forms, but courts think they are better. (problem is still subjectivity) • Performance should be under the control of the employee • Should relate to specific functions of the job, not global assessments (#2 could be a problem for a behavioral measurement system if used for personnel decisions and no job analysis)

Common types of appraisal methods (NFE) • Graphic rating scales, most common • Employee comparison methods • Evaluate employees by comparing them against each other rather than against a standard • Behavioral checklists and forms

SO3A: Employee comparisons vs. graphic rating scales • Advantage of employee comparison over graphic rating scales? • Forces variance or variability into the ratings • With graphic scales, not uncommon for a supervisor to rate employees pretty much the same: everyone is a 5 or a 6 on a 7 point scale, making it difficult to make personnel decisions.

SO3B: What’s the problem with employee comparisons? • May create a false impression that differences between individuals are actually larger than they are • Six employees, rank ordered • Jan • Shakira • Paul • Mike • Susan • John Actual performance difference between Jan and Mike or even Susan may be very small, yet look big. Can affect salary increases, promotions, etc.

Dissatisfaction with performance appraisal • Companies tend to switch frequently from one type of form to another • Recall performance appraisal was the third-ranked topic addressed in JAP • IBM abandoned employee ranking in favor of graphic scales, then returned to ranking a few years ago • Made the front page of the Wall Street Journal: ranking was THE way to go • Why do you think IBM may have made the switch? (nothing to do with reliability or validity)

SO4: Behavioral checklists • Behavioral checklists and scales are assumed to be more accurate because they are less vague, but they are not. • Why? • Still are based on subjective judgments • Completed once a year • That is also why we should not consider them adequate from a behavioral perspective • Objective measures of behavior/performance • Over time as behavior/performance occurs on the job (aubrey daniels, problems, do it 1/2 as often)

SO6: Main problem with performance appraisals (NFE) • Inflation of ratings • Main assumption about the cause of errors • Design of the form (hence why so much time and money is spent redesigning forms - “design out” rating errors • Lack of knowledge on the part of supervisors/managers - training • Rarely are the consequences for accurate ratings examined or considered (which leads to SO7) (Important lead-in to SO7, Director of Personnel, WMU, ranks among the worst problems she has - fire/good ratings)

SO7: Consequences for inflated ratings (learn 3, name of principle for 4) • No rewards for accuracy; no or few sanctions for inaccurate ratings (how does the organization know if the ratings are accurate or inaccurate - a problem) • Most common reason - high ratings are necessary to get salary increases, promotions and other rewards for employees • Ratings received reflect their competence as a supervisor/manager

4th consequence • Negative evaluations result in defensive reactions and hostile reactions from employees • Principle of behavior? Straightforward avoidance: In past: Bad rating ––––> SP aversive social interaction CS ––––––> CR stress Current: Good rating –––––> Sr-, no aversive social interaction No CS to elicit CR of stress (on click)

SO8 Self-assessments • SO8A: Main problem? • Inflation: we think we are better than we are • Engineer example: 92 engineers rated their performance in comparison to other engineers. On average, each engineer rated his performance as better than 75% of the other engineers. Quite a feat to have 100% of your employees in the top 25%!

SO8 Self-assessments, cont. • SO8B: Thornton - little agreement between supervisory and self-assessments. What are the very important implications of this from a behavioral perspective? In behavior analysis we always strive to make rewards contingent upon performance. The better the performance, the more rewards. If, however, employees believe they are performing better than the supervisor believes, then the supervisor will not give them the rewards that they feel they should be getting. That is, employees will not believe that rewards are truly contingent upon their performance. Likely to hurt performance and cause dissatisfaction.

SOs 9&10: Peer assessments • Peer assessments are very accurate - high reliability and high validity • Reliability - different people rate the employee the same way • Validity - related to job performance • Main problem? • Acceptance by employees - we don’t like to do this • WMU faculty merit system: Union vs. management • Not enough money to go around; many just split $$ • How would you like to be given the responsibility for evaluating your peers - the other students advised by your faculty advisor - and have rewards (i.e.,assistantships, grades in classes, opportunities for practica and projects) based on that? (we may not know how well we are performing, but we know how well our peers are performing!)

SO11: 360 feedback: hot, hot, hot • A manager/supervisor • Rates himself • Is rated by his manager/supervisor • Is rated by his subordinates • SO11A: What percentage of managers saw themselves as others saw them? Only 10%! • SO11B: Overrating was most noteworty on what scale? People (NFE, but why is this good news for us?)

SO12: Credibility & Power - Fisher and Taylor • Credibility: What influences credibility the most from a behavioral perspective? (assuming manager has expertise) • Measurement system and its objectivity • Judi Komaki - work sampling separates highly effective managers from ineffective ones (actually objectively sampling and looking at work) • If the supervisor is not evaluating performance accurately, the feedback becomes meaningless • Daniels “In God we trust, all else bring data!!” (important - ilgen, fisher & taylor, the extent to which people will change their performance when given fb from supv depends upon)

SO12B: Power and feedback • Power: The extent to which a supervisor has control over valued rewards • Why does power influence the extent to which an employee will be influenced by a supervisor’s feedback? (this is the question) Feedback is not a basic principle of behavior. It is just a stimulus. It will be a neutral stimulus unless it is paired with valued consequences. When it affects behavior, it most likely does so as an SD or Sr (or an analog). But, in both cases to become an SD or Sr, it must be paired with valued consequences. Thus, if feedback is not paired with valued consequences (the supv. has no control over them), it will not affect behavior. (excellent point from a behavioral perspective, students have trouble with this, a nice behavioral analysis)

For your entertainment only: Actual statements • His men would follow him anywhere, but out of morbid curiosity • I would not allow this employee to breed • Works well when under constant supervision and cornered like a rat in a trap • He would be out of his depth in a parking lot puddle • This young lady has delusions of adequacy • This employee is depriving a village somewhere of an idiot

End Part 1 • Any questions so far?

Unit 2: Performance Assessment & Measurement