Important Issues in Defining, Measuring and Using Performance Measures

1. Seminar on Performance Appraisal: Important Issues in Defining, Measuring and Using Performance Measures Ahmad Zhafir bin Zulkfli @ Zulkifli Nadzirah binti Mohamad Yusof Syarifah Nur Hanisah binti Tuan Simbak

2. Outline • Introduction Of Issues In Performance Measure • Issues In Measuring Performance In Healthcare • Using Performance Measures Effectively • Conclusion

3. Performance appraisal in healthcare organization • Performance appraisal is a required process in healthcare organizations to ensure that the quality of care is met. This process uses methods to provide employees with the information necessary to determine whether they are meeting expectations or can do better to improve their performance (Huber, 2006)

4. Problems in Performance Appraisal (Giangreco, Carugati,Sebastiano, & Tamimi, 2012).

5. Main Problems Faced by Nurses in Performance Appraisal Contextual Problems Structural Problems of the Appraisal System Problems in the Appraisal Process Problems Related to Appraisal Results Nikpeyma, N., Abed-Saeedi, Z., Azargashb, E., & Alavi-Majd, H. (2014). Problems of clinical nurse performance appraisal system: A qualitative study. Asian Nursing Research, 8(1), 15–22. https://doi.org/10.1016/j.anr.2013.11.003

6. Ndlovu et al. (2024),

7. A study by Mok and Leong (2021) was conducted in private hospitals in Malaysia to assess the effectiveness of employee performance appraisals. What did they find? • . The appraisal process and the perception of fairness play key rolesin whether the evaluation is seen as effective. However, they also found that subjective factors such as: • Trust between employee and supervisor • Communication style These can influence appraisal results. These factors are difficult to measure objectively and may vary depending on the supervisor’s personal opinion.

8. It highlights conceptual challenges: • What exactly is considered “good performance”? • How can we measure “communication” or “attitude” fairly? • Are appraisals really based on actual job performance, or more on personal opinions? • Conceptual challenges refer to problems or difficulties in understanding, defining, or applying core ideas within a system. In the context of performance measures, they are the issues that arise when trying to design or use evaluation methods that are fair, meaningful, accurate, and useful.

9. Defining Performance Measures • Issue: Performance measures vary by setting • Health service performance measures must reflect the context—what works for a tertiary hospital may not suit a rural clinic. • Example: A KPI like “30-day surgical mortality” fits hospitals, while “antenatal visit coverage” suits community clinics. • In hospitals, indicators might focus on surgery success rates or patient mortality. • In clinics, measures might include waiting time or patient satisfaction. • In public health, focus shifts to population-level metrics like immunization coverage or disease incidence. Smith et al. (2009). Performance Measurement for Health System Improvement (WHO Europe).Ministry of Health Malaysia. (2023). Annual Report 2023.

10. Must reflect organizational objectives, not just efficiency: • If your goal is quality care, measuring only cost efficiency creates misalignment. • Focusing on reducing consultation time might improve throughput but worsen patient satisfaction. • OECD. (2024). Rethinking Health System Performance Assessment.Mannion & Davies (2008). Public Money & Management.

11. Issues in Measuring Performance in Healthcare “The ideal way to deal with poor performance is not to let it happen in the first place!”

12. Performance measures criteria Reference: Buchbinder, S. B., Shanks, N. H., & Kite, B. J. (2021). Introduction to health care management (4th ed.). Jones & Bartlett Learning.

13. Use case: Malaysian Civil Workers HRMIS

14. SasaranKerjaTahunan (SKT) & LaporanPenilaianPrestasiTahunan (LNPT)

15. MyPerformance module

16. Important performance measurement issues

17. 1. Contrast-effect bias • A worker’s performance would be judged in contrast to the workers around them • The contrast effect occurs when the evaluation of a current performance is influenced by the performance of others • For example, an average performer may be rated lower if they follow a high performer, and higher if they follow a low performer Reference: Borkowski, N., & Meese, K. (2021). Organizational Behavior in Health Care (4th ed.). Jones & Bartlett Learning. Palmer, J. K., & Gore, J. S. (2014). A Theory of Contrast Effects in Performance Appraisal and Social Cognitive Judgments. Psychological Studies, 59(4), 323–336. https://doi.org/10.1007/s12646-014-0282-6

18. 2. Positive and negative halo effect • Positive halo effect: Evaluation error that occurs when a manager generalizes one positive performance feature or incident to all aspects of employee performance, resulting in a higher rating • Negative halo effect/horn error: one negative performance resulting in a lower rating • For example: a HOD noted that one of his MO was a neat individual, therefore he gave her high marks on this factor. Conscious or unconsciously, he also gave her good marks on the criteria that she underperformed, or otherwise Reference: Mondy, R. W., & Martocchio, J. J. (2022). Human Resource Management (14th ed.). Pearson Education Limited.

19. 3. Multisource feedback limitations • Peer feedback included in 360-degree evaluation may also be biased or inaccurately given, due to difficulties in determining an individual’s contribution to the unit or service • Fear of providing negative feedback to a colleague • Negative reactions are also a possibility • An organization and its members must demonstrate a willingness and readiness for the process Reference: Buchbinder, S. B., Shanks, N. H., & Kite, B. J. (2021). Introduction to health care management (4th ed.). Jones & Bartlett Learning.

20. 4. Variability in appraisal methods • Different appraisal methods are used across various ranks and departments, including behavioral approaches (rating scales, peer rating) and outcome approaches (management by objectives) • Whether to evaluate traits, behaviors or outcomes of work • 180 degrees vs. 360 degrees appraisal • This variability can lead to inconsistencies in performance measurement Reference: Sing, R. D. R., & Vadivelu, S. (2019). A Study on Employee Performance Appraisal System in Hospitals—With Respect to Hospitals in Vellore City. Indian Journal of Public Health Research & Development, 10(6), 233. https://doi.org/10.5958/0976-5506.2019.01273.7

21. 5. Invalid and unreliable measures • Performance appraisals often use measures that are not valid or reliable, leading to inaccurate assessments of individual performance • Most health care delivery functions are group performance • For example: patient outcome at the emergency department depends on the performance of all team members in that shift • Using patient outcome to measure performance of individual physician are thus unreliable due to other factors outside the control of physicians such as specific disease mortality rate, patient compliance, consults with other physicians Reference: Labig, C. E. (2008). Bad Measures Don’t Make Good Medicine: The Ethical Implications of Unreliable and Invalid Physician Performance Measures. Journal of Business Ethics, 88(2), 287–295. https://doi.org/10.1007/s10551-008-9963-1

22. 6. Leniency or strictness • Raters sometimes place every employee at the extreme high or low end of the scale • Often motivated by a desire to avoid controversy over the appraisal • Most prevalent when highly subjective performance criteria are used • Legal consideration: firing problematic employee with recent excellent performance rating! Reference: Mondy, R. W., & Martocchio, J. J. (2022). Human Resource Management (14th ed.). Pearson Education Limited.

23. To overcome: SMART indicators Reference: Selvik, J. T., Bansal, S., & Abrahamsen, E. B. (2021). On the use of criteria based on the SMART acronym to assess quality of performance indicators for safety management in process industries. Journal of Loss Prevention in the Process Industries, 70(1), 104392.

24. Constructive Dismissal & Performance Appraisals Thomas Kuruvilla v. MDEC (2021) • Claimant had been with MDEC for 10 years, and had no prior performance issues • New CEO was appointed, a reorganization took place, and the Claimant was transferred to various new departments, placed on PIPs, and then dismissed on the grounds of being a poor performer • The Court also found that there had been “delays in setting the KPIs, thereby reducing the period for its achievement” and “the extent of his duties had not been properly defined and demarcated” • The Court concluded that the Claimant was dismissed without just cause or excuse. The Court ordered MDEC to pay the Claimant RM810,628.00 (24 months’ back wages, plus 10 months’ salary as compensation in lieu of reinstatement for 10 years of service) Reference: Geyzel, M. van. (2021, September 9). Employee poor performance: Some recent cases. The Malaysian Lawyer. https://themalaysianlawyer.com/2021/09/10/employee-poor-performance-recent-cases/

26. References • Borkowski, N., & Meese, K. (2021). Organizational Behavior in Health Care (4th ed.). Jones & Bartlett Learning. • Buchbinder, S. B., Shanks, N. H., & Kite, B. J. (2021). Introduction to health care management (4th ed.). Jones & Bartlett Learning. • Geyzel, M. van. (2021, September 9). Employee poor performance: Some recent cases. The Malaysian Lawyer. https://themalaysianlawyer.com/2021/09/10/employee-poor-performance-recent-cases/ • Labig, C. E. (2008). Bad Measures Don’t Make Good Medicine: The Ethical Implications of Unreliable and Invalid Physician Performance Measures. Journal of Business Ethics, 88(2), 287–295. https://doi.org/10.1007/s10551-008-9963-1 • Mondy, R. W., & Martocchio, J. J. (2022). Human Resource Management (14th ed.). Pearson Education Limited. • Palmer, J. K., & Gore, J. S. (2014). A Theory of Contrast Effects in Performance Appraisal and Social Cognitive Judgments. Psychological Studies, 59(4), 323–336. https://doi.org/10.1007/s12646-014-0282-6 • Selvik, J. T., Bansal, S., & Abrahamsen, E. B. (2021). On the use of criteria based on the SMART acronym to assess quality of performance indicators for safety management in process industries. Journal of Loss Prevention in the Process Industries, 70(1), 104392. https://doi.org/10.1016/j.jlp.2021.104392 • Sing, R. D. R., & Vadivelu, S. (2019). A Study on Employee Performance Appraisal System in Hospitals—With Respect to Hospitals in Vellore City. Indian Journal of Public Health Research & Development, 10(6), 233. https://doi.org/10.5958/0976-5506.2019.01273.7

27. Using Performance Measures Effectively

28. Characteristics of an Effective Performance Appraisal System (Mondy & Martocchio, 2016)

29. Characteristics of an Effective Performance Appraisal System (Mondy & Martocchio, 2016)

30. Issues in Usage of Performance Appraisals Kaplan & Norton, 1996; Gulzar et al., 2020

31. Best Practices for Use of Performance Appraisals

32. Case study

33. Distribution of Measures by BSC Perspective

37. Methodology

38. 📊 Summary: Identification and Ranking of BSC Measures in Private Hospitals Initial measures considered: 44 Final accepted measures: 31 Financial Perspective: 10/11 (91%) Customer Perspective: 9/13 (69%) Internal Business Process (IBP): 7/11 (63%) Learning & Growth: 5/9 (55%)

39. 📊 Summary: Ranking & Weighting of BSC Performance Measures 📈 Ranking Results: • AIS values range: 4.2 to 4.9 (out of 5) • Top-ranked indicators (AIS = 4.9): • Number of Patient Complaints(Customer perspective) • Medical Error Rate(Internal Business Process perspective) • Lowest-ranked: Employee Satisfaction Index (AIS = 4.2)

40. Key Contributions of the Study:

41. Recommendations

42. Conclusion

43. 📚 References • Behrouzi, F., & Ma'aram, A. (2020). Identification and ranking of specific balanced scorecard performance measures for hospitals: A case study of private hospitals in the Klang Valley area, Malaysia. The International Journal of Health Planning and Management, 35(6), 1534–1552. https://doi.org/10.1002/hpm.3039 • Gulzar, F., Javaid, M. M., & Naeem, H. (2020). Performance appraisal system: A study on employees’ central issues and challenges in healthcare. Journal of Organisation & Human Behaviour, 9(1), 15–27. • Kaplan, R. S., & Norton, D. P. (1996). The balanced scorecard: Translating strategy into action. Harvard Business Press. • Bevan, G., & Hood, C. (2006). What’s measured is what matters: Targets and gaming in the English public health care system. Public Administration, 84(3), 517–538. • Smith, P. C., Mossialos, E., Papanicolas, I., & Leatherman, S. (2009). Performance measurement for health system improvement: Experiences, challenges and prospects. World Health Organization. https://apps.who.int/iris/handle/10665/107921 • Mondy, R. W., & Martocchio, J. J. (2016). Human resource management (14th ed.). Pearson Education.