Stephen Charles MA, MS, PhDc Dawn Schocken, MPH, PhD

1. What is the impact of a certificate program for standardized patients on the end of year assessments? Stephen Charles MA, MS, PhDc Dawn Schocken, MPH, PhD

2. What is the impact of a Certificate Program for Standardized Patients on the end of year assessments? Background: • A trained standardized patient (SP) provides student feedback to enhance student's communications skills. • Feedback is core to the student's understanding of effective patient management. • Inter-observer reliability and validity among SPs in these encounters is key to the standardization of feedback. • This study tracked the reliability and validity among SPs in MS III end of year assessments.

3. What is the impact of a Certificate Program for Standardized Patients on the end of year assessments? Background: • Inter-Observer Reliability and Validity is defined as an independent observer of a film being able to complete a checklist within a single standard deviation as the original rater. • The initial rater is the Director of the Center and Trained Faculty Observers. Additional rater reliability is validated with Industrial Psychologists with psychometric training and expertise.

4. What is the impact of a Certificate Program for Standardized Patients on the end of year assessments? Summary of work: • 35 SPs took the ninety hour Certificate Program. • Using active learning to deliver these unique sessions, each SP was filmed pre and post training. • Debrief both individual and group enhanced performance. • Analysis was completed of recorded student encounters detailing inter-rater reliability.

5. What is the impact of a Certificate Program for Standardized Patients on the end of year assessments? Summary of work: • The Standardized Patients receive the following: • 30 Hours of Educational Theory, Communication Skills and Feedback / Evaluation Methods Coursework. • 30 Hours of Physical Examination Skills Coursework • 15 Hours of Film Review and Evaluation • 15 Hours of Mentorship of SPs with less seniority

6. What is the impact of a Certificate Program for Standardized Patients on the end of year assessments? Summary of results: • Initial data suggest improvement in inter-rater reliability among these SPs. • Review of MS III year end assessment showed in a comparison of the cohort to traditional SPs, 50% of the cohort increased their inter-rater reliability while 70% of the regular SPs showed a decrease. • The inter-rater reliability was up a significant 87% over last year’s comparison.

7. What is the impact of a Certificate Program for Standardized Patients on the end of year assessments?

8. What is the impact of a Certificate Program for Standardized Patients on the end of year assessments?

9. What is the impact of a Certificate Program for Standardized Patients on the end of year assessments? Conclusions: • Targeted training enhanced the inter-observer reliability and validity of the SP/Student encounter. • Future studies will include a self analysis of SP responses to student encounters.

10. What is the impact of a Certificate Program for Standardized Patients on the end of year assessments? Take-home messages: • Inter-rater reliability assures a more standardized assessment of student performance.

11. What is the impact of a Certificate Program for Standardized Patients on the end of year assessments? • Reference Slides to Follow

12. What is the impact of a Certificate Program for Standardized Patients on the end of year assessments? Certificate Program Outline • Orientation to Standardized Patient Activities – Initial 30 Hours Training • Overview of Organ Systems / Exams – PE Training – (3 hours) • Practical of Organ Systems Exams – PE Training (Each of nine organ systems review at 3 hours each) • Overview of Communications Skills – Initial 30 hours of training • Offering Feedback to Students – Initial 30 hours of training • Overview of Case-Based Training – Initial 30 hours of training • Practical / Debrief of Case-Based Trials – Initial 30 hours of training • The Art of Dealing with Instructors – Initial 30 hours of training • Acting Anyone? – Initial 30 hours of training • Overview of Specialty Training: • Breaking Bad News – Initial 30 hours of training • Hybrid Cases / Confederate Cases – Initial 30 hours of training • Female GYN Specialization – Outside of Certificate Training • Male GU Specialization – Outside of Certificate Training • Standardized Grading – Initial 30 hours of training