Developing and Validating an Assessment Measure

1. Developing and Validating an Assessment Measure

2. Goals, Objectives & Criteria • It is critical that employees have a clear understanding about what part of their performance is appraised and how it will be measured.   • Performance can be measured according to previously established and mutually understood goals between the employee and the supervisor. • Goals should be challenging but attainable. • Goals must be related to the core responsibilities. • After the goals and objectives are established, criteria against which the employee's job performance can be evaluated are agreed upon. Criteria are the indicators of successful performance on the job.

3. Good performance goals, objectives, and criteria are: • job related • challenging, but achievable • clearly communicated & mutually understood • specific and objective • time-oriented • written • subject to revision, as needed

4. Six Steps (1) conducting a job analysis (2) developing predictors (3) developing criterion measures (4) administer predictor & criterion measures (5) conducting validation & fairness analyses (6) implementation

5. Examples • Let’s review a few cases • http://www.pdri.com/value/assess

6. The errors ‘we’ make

7. Halo/Horn Error • Halo error - Occurs when supervisor generalizes one positive performance feature or incident to all aspects of employee performance resulting in higher rating • Horn error - Evaluation error occurs when supervisor generalizes one negativeperformance feature or incident to all aspects of employee performance resulting in lower rating

8. Leniency/Strictness • Leniency - Giving undeserved high ratings • Strictness - Being unduly critical of employee’s work performance • Worst situation is when ‘institution’ has both lenient and strict supervisors and does nothing to level inequities

9. Central Tendency • Error occurs when employees are incorrectly rated near average or middle of scale • May be encouraged by some rating scale systems requiring evaluator to justify in writing extremely high or extremely low ratings

10. Recent Behavior Bias • Employee’s behavior often improves and productivity tends to rise several days or weeks before scheduled evaluation • Only natural for rater to remember recent behavior more clearly than actions from more distant past • Maintaining records of performance

11. Personal Bias (Stereotyping) • Supervisors allow individual differences such as gender, race or age to affect ratings they give • Effects of cultural bias, or stereotyping, can influence appraisals • Other factors • like an individual that may be appraised more harshly simply because they do not seriously object to results

12. Manipulating the Evaluation • Sometimes, supervisors control virtually every aspect of appraisal process and are in position to manipulate system • Example: Want to give pay raise to certain employee. Supervisor may give employee an undeserved high performance evaluation

13. Employee Anxiety • Evaluation process may create anxiety for appraised employee • Opportunities for promotion, better work assignments, and increased compensation may hinge on results

14. Establishing Performance Criteria (Standards) • Traits • Behaviors • Competencies • Goal Achievement • Improvement Potential

15. THREE FOCAL POINTS OF APPRAISAL 1. PERSONAL TRAITS AND CHARACTERISTICS + inexpensive to develop and use + not specialized by position; one form for all workers - high potential for bias and rating errors - not very useful for feedback or development - not easily justifiable for reward/promotion decisions 2. JOB BEHAVIOR AND ACTIVITY + can focus on specific duties listed in the job description + intuitively acceptable to employees and superiors + useful for providing feedback + seem fair for reward and promotion decisions - are time consuming to develop and use - can be costly to develop - have some potential for rating error and bias

16. THREE FOCAL POINTS OF APPRAISAL 3. WORK RESULTS AND OUTCOMES + less subjectivity bias + acceptable to employees and superiors + links individual performance to organizational objectives + seem fair for reward and promotion decisions - are time consuming to develop and use - may encourage a short-term perspective - may use deficient or inappropriate criteria

17. GOALSETTING ISSUES GOAL DIFFICULTY – How challenging should the work objectives be? I want an easy goal, but the organization wants me to “stretch.” ACCEPTANCE – Will workers feel committed to work toward objectives that have been assigned to them, rather than those set participatively? SPECIFICITY – Precise quantitative indicators may not exist for critical elements of the job. General, open-ended goals are not easily assessed. MOTIVATION – Objectives should be challenging, yet reachable. They also need to be linked to desirable rewards to successfully motivate workers.