Prepare and Use Skills Assessments

1. Prepare and Use Skills Assessments

2. Introduction • Why do we assess student clinical skills? • What is challenging about preparing and using skills assessments?

3. Steps in Skill Development • Introduce and demonstrate a skill. • Observe students as they practice the skill and give feedback to help them improve their performance. • Assess students for competency in the skill.

4. Objectives • Select methods for assessing the skills of students • Prepare skills assessments • Develop structured practical examinations • Administer and score skills assessments • Use results to improve performance

5. Skill Assessment Methods • Direct observation of students as they perform skills • Structured feedback reports on students’ performance • Logbooks, learning journals, and care plans

6. Direct Observation #1 • Most valid way to assess students’ skills. • Can be used for both formative and summative assessments. • Need to be creative developing approaches when you have large numbers of students.

7. Direct Observation #2 • Oral questioning can be used with direct observation. • Can be done with simulations (formative) and with patients (summative).

8. Structured Feedback Reports #1 • Assessing sustained performance rather than just taking “a snapshot” as you would with an examination. • Can cover overall performance, demonstrated attitudes, and essential healthcare delivery skills. • Useful for assessing characteristics such as personal attributes, attitudes, and professional values.

9. Structured Feedback Reports #2 • Are easy, efficient, and consistent • Provide a formal structure for assessment, particularly formative assessment • Reinforce essential skills • Ensure that each student receives feedback • See Sample 11-1 and 11-2

10. Logbook • The logbook (also called a casebook) contains a list of skills or tasks that students should be able to perform. • The students are responsible for learning how to do each of the tasks, and when they believe they are ready, they can ask a teacher, tutor, or clinical instructor to assess their performance.

11. Learning Journal A learning journal is used to record learning experiences, especially those in which the student has minimal or no supervision, such as home visits, community-based experiences, or rotations to distant clinic sites.

12. Care Plan • A care plan is used to document the patient’s problems, care required, and expected outcomes. • Students are often required to create care plans to demonstrate their understanding of and ability to explain management required for a specific problem.

13. Checklists • A list of steps needed to perform a skill correctly, listed in the correct sequence. • Assessor observes each step. • Well-constructed checklists should contain only sufficient detail to help the assessor evaluate and record the student’s performance.

14. Designing a Checklist #1 • Identify the steps or tasks: • Adapt an existing tool, or • Conduct a task analysis • Place the steps in the correct sequence. • Identify the standards or minimum level of performance (Yes/No or Multi-level rating systems).

15. Designing a Checklist #2 • Include the key elements of a checklist (title, space for names, course information, instructions, space for signature). • Field-test the checklist. • See Samples 11-3, 11-4 and 11-5.

16. Structured Practical Examinations #1 • Typically students rotate through a series of stations where they answer questions (orally or in writing), or perform tasks while being observed. • Students may demonstrate a skill, interpret diagnostic materials, or respond to short questions or case studies.

17. Structured Practical Examinations #2 • This type of examination is also known as a multiple station assessment test (MSAT). • The Objective Structured Clinical Examination (OSCE) is one of the most well known forms of the structured practical examination.

18. Components of a Structured Practical Examination • All students rotate through multiple stations and are tested on the same KSAs. • There is a time limit for each station. • An assessor at each station that requires observation. • All students are assessed according to the same standards.

19. Steps to Prepare for a Structured Practical Examination #1 • Choose which learning objectives will be tested. • Decide on a problem, issue, or activity that addresses each learning objective. • Map out a plan for the stations (see Sample 11-1).

20. Steps to Prepare for a Structured Practical Examination #2 • Plan the details for each station: • Write the task/scenario to be completed. • Develop instructions. • Develop assessment tools. • List the resources needed.

21. Before the Skills Assessment • Discuss previous practice sessions with the student. • Ask if the student has any questions about the skill and is ready to be assessed. • Review the assessment tool.

22. During the Skills Assessment • Observe and assess the student’s performance: • Stand where you can see without intruding and let the student perform the skill. • Do not interfere unless the student is about to make a mistake that may endanger the patient. • Provide only essential feedback while the student is performing the skill.

23. After the Skills Assessment • Review the skill with the student (student shares what she or he did well and what could be improved). • Provide positive feedback and offer suggestions for improvement. • Determine if the student is competent or needs additional practice.

24. Use Results to Improve Performance #1 • Give students an opportunity to ask you questions about steps they did not understand or they performed incorrectly. • Instruct students to practice the steps that they performed incorrectly.

25. Use Results to Improve Performance #2 If many students had trouble with the same tasks, either the teaching methods or materials did not adequately cover that learning objective, or the task needs to be redefined.

26. Summary #1 • Direct observation is the most valid method for assessing skills, but can be influenced by the judgment of the observer. • You can improve this method by using standardized tools such as checklists to guide assessment.

27. Summary #2 • Structured practical examinations require time for planning and preparing valid stations, but provide a highly structured and reliable method for assessing knowledge, skills, and attitudes that can be used from year to year. • Assessing students’ skills is one of the keys to improving students’ performance.

28. Discussion • In the large group or in small groups: • What is the role of the EDC, PIC and QCC in assisting with student knowledge and skills assessment? • What challenges do we face in strengthening student knowledge and skills assessments?