Holistic Assessment of Social Work Students’ Professional Practice Marion Bogo

1. Holistic Assessment of Social Work Students’ Professional PracticeMarion Bogo Joint Social Work Education Conference and Social Work Research Conference University of Hertfordshire 2010

4. Overview • Link between social work education and practice • Conceptualization of professional practice • Assessment • Cognition • Interpersonal • Contextual

5. Social Work Education and Practice The purpose of social work education is to prepare social work practitioners

6. Professional Work:Dimensions • Thinking • Performing • Acting with integrity (Lee Shulman, 2005)

7. Signature Pedagogy “…pedagogical norms with which to connect and integrate theory and practice.” (Lee Shulman, 2005)

8. Social Work Education and Practice • United States: Council on Social Work Education, Educational Policy and Accreditation Standards 2008 • England: Social Work Task Force 2009 • Canada: Canadian Council of Social Work Regulators; Project - Social Worker Competency Profile 2010

9. Social Work Education and Practice

10. Conceptualization of Professional Practice • Critique of competency as dimensions and skills – a checklist approach • How to capture holistic nature of practice and performance • not mechanistic • context • professionalism • knowledge competencies, internal cognitive processes for judgment, decision making

11. “In their own words…” Design a practice-based evaluation tool • Reflect practice teachers’ tacit or implicit conceptualization of student competence – dimensions they use • In practice teachers’ language

12. Holistic Competence

13. Meta-Competencies • Higher order overarching abilities and qualities - of a different order and nature than procedural or operational behaviors and skills. • Related to individuals’ ability to: • use discrete behaviors in a purposeful, integrated, and professional manner. • learn the specific role competencies of particular professions.

14. Behavior in the Organization Conceptualization of practice Collaborative relationship Meta-competencies Relational capacity/Intentional use of self Procedural competencies Conducts assessment and intervention Professional Communication Learning and growth as a professional DIRECT PRACTICE

15. Behavior in the Organization/ Relationships Critical Thinking and Analysis Planning Meta-competencies Leadership Procedural competencies Project management Professional Communication Learning and growth as a professional MACRO PRACTICE

16. Social Work Education and Practice

17. Challenges to Assessment

18. Levels of Performance • High agreement on individual rankings of 20 vignettes- interclass correlation coefficient: 0.83 • Five categories generated • Exceptional - - - - - - - - - - - - - - - - - 5 • Ready for practice - - - - - - - - - - - - 4 • On the cusp - - - - - - - - - - - - - - - - - 3 • Needs more training - - - - - - - - - - -2 • Unsuitable for the profession - - - - 1

19. From Descriptions to a Tool • Dimensions of holistic competence • Learning and growth • Behavior in the organization • Conceptualizing practice • Collaborative relationships • Assessment and intervention • Professional Communication

20. From Descriptions to a Tool • For each dimension • Extracted descriptors from the ranked vignettes for each of 5 levels of performance • In practice teachers’ language.

21. Behavior in the Organization

22. Competency-Based Evaluation

23. Results New PBE tool was not more effective at discriminating students than older CBE tool.

24. Using all Tools • Recall most recent student • 20 vignettes ordered randomly • Select vignettes “similar” to their student. • Select from “similar vignettes” one or two vignettes that are “most similar” to their student. • Evaluate same student using • Practice-Based Evaluation (PBE) Tool • Competency-Based Evaluation (CBE) Tool

25. Distribution of Student Scores for the Three Measures

26. Distribution of Student Scores for the Three Measures

27. Distribution of Student Scores for the Three Measures

28. Scales • Holistic impression of students • Deconstruct • Relate to a skill • Convert to a point on a scale

29. Challenges to Assessment

30. Reflections on Assessment • When values collide • Student response and the inter-personal relationship • Negotiating • Where is ‘the school?’

31. Conclusion • In instructors’ own language – holistic, global, authentic -reflects their conceptualization of practice competence. • A tool without numbers. • Approaches that empower the practice assessor/field instructor and are not negotiated – exercise her expertise.

32. An Online Tool

33. Dimensions of Practice • Six dimensions • Learning and growth • Behavior in the organization • Clinical relationships • Conceptualizing practice • Assessment and intervention • Professional Communication • For each dimension, descriptors provided for each of 5 levels of performance.

34. Levels of Performance Five categories • Exceptional - - - - - - - - - - - - - - - - - 5 • Ready for practice - - - - - - - - - - - - 4 • On the cusp - - - - - - - - - - - - - - - - - 3 • Needs more training - - - - - - - - - - -2 • Unsuitable for the profession - - - - 1

35. Conceptualizing Practice

36. Conceptualizing Practice

39. 2008-9 Comparisons between CBE Tool (480) and PBE Tool (152)

40. Challenges to Assessment

41. OSCE: Objective Structured Clinical Examination • Developed in medicine and used in health professions (Harden et al., 1975) • Students interview series of standardized, simulated “clients” • “Clients” are trained actors • Students are given brief information – role, presenting issues, time frame and goal.

42. Adapted for Social Work • Holistic competence - in social work two components: • interview for 12-15 minutes for procedural competencies • followed by a structured reflective dialogue for 12-15 minutes to elicit meta-competencies • Both observed and rated by a trained rater.

43. Iterative Process to Develop OSCE

44. The Examination

45. Reading Instructions

46. One of Five “Clients”

47. Hospital Social Work

48. Child Protection

49. Rater and Widowed Mother

50. Initial Study: Participants • 11 students completed Year 1 of the MSW Program • 7 recent graduates of the MSW Program • 5 experienced social workers with 5-7 years of experience