Outcomes of Social Work Education (OSWE): Findings from the Projects

1. Outcomes of Social Work Education (OSWE): Findings from the Projects Hilary Burgess, John Carpenter, Joanna Fox, Roxana Anghel, Sharon Vitali, Juliet Koprowska, Kish Bhatti-Sinclair, Anne Quinney, Imogen Taylor, Clare Ockwell, Suzy Braye, Michelle Lefevre

2. Outline of Session • Introduction to OSWE project • Presentation of findings from six partners in England with questions for clarification. • Discussion of the findings in general and their implications. • Key issues in doing outcomes-focussed research on Social Work Education. • Open discussion.

3. Aims of OSWE (3 year project) • To test the feasibility of outcome measures and research designs in Social Work Education • To generate high quality evidence about the effectiveness of methods of SWE • To build capacity and capability amongst academics, including service users • To use opportunities to compare and contrast practice between programmes

4. A Collaborative Capacity & Capability building model • Peer learning through Action Learning Set • Support, advice and mentoring, F2F, e-mail and by phone (e.g. data analysis). See Burgess, Hilary and Carpenter, John (2008) 'Building Capacity and Capability for Evaluating the Outcomes of Social Work Education (the OSWE Project): Creating a Culture Change', Social Work Education.

5. Measuring Outcomes • Measuring changes over time • Before-after designs • Using and adapting measures

6. Levels of Outcome 1.Learners’ reactions 2.Modifications in attitudes and perceptions Attitudes Motivational 3.Acquisition of knowledge and skills Procedural Strategic knowledge Initial skills Compilation skills 4.Changes in Behaviour 5.Benefits to users and carers Focus Attitudes to race and racism (Soton) Attitudes interprofessional practice (Sussex) Understanding partnership with SU&C (Anglia Ruskin) Communication skills with children (Sussex) Interviewing and communication skills (York) Use of research skills (Bournemouth) Acquisition of Soc. Wk. competences (Oxford Brookes) Primary levels of Outcomes(after Carpenter, 2005)

7. The outcomes of teaching and learning about ‘race’ and racism Kish Bhatti-Sinclair Division of Social Work Studies School of Social Sciences University of Southampton

8. Aims and Methods • Students’ understanding and experience of ‘race’ and racism; and • How ‘race’ and racism is addressed in the degree curriculum Questionnaires to BSc social science students in week 2 of Semester 1 (153 students) and in Week 8 (71 students) of Semester 2. 34 respondents completed both sets. Social work students = 41.2%.

9. TYPES OF LEARNING Seminars 17 Course units 16 Personal study 9 Shadowing soc wkr 4 Peer Group 3 Personal tutors 2 SPECIFIC COURSE UNITS 2 out of total of 8 (3 SW) Social problems and social policy - 6 Sociology of everyday life - 2 Main sources of learningabout race and racism (frequency)

10. Results • A small increase in self-rated knowledge of racism – 10%. • A increase in respondents’ confidence in challenging racism (but not statistically significant.) • Overall results suggest that formal teaching about ‘race’ and racism did influence knowledge although the impact was small.

11. Limitations • Sampling – not random. • Results not generalisable – must be interpreted with caution. • Knowledge of racism based on self rating - not possible to obtain an objective measure of knowledge.

12. Outcomes of “Using Research for Practice” using blended learning. Bournemouth UniversityAnne Quinney BA Social Work Year 2 students.

13. Methodology Pre-Post tests Measure: Research Self Efficacy (RSE) scale (Holden et al., 1999). Subscales on research knowledge and skills. PLUS 5 items “using computer and information technology”. Analysis: Paired t-tests for difference in mean scores.

14. Findings: Research Knowledge and Skills “How confident are you that you can successfully analyse basic quantitative and qualitative data?” Scale 0-10

15. Findings: using computer and information technology “How confident are you that you can successfully access research findings from research bodies, social work organisations, government departments etc (e.g. JRF, SCIE) using the internet?”

16. Next Steps and Uses • Do Research Self Efficacy scores and assignment marks correlate? • Compare with another programme • Uses: • Adjust the curriculum for (1) present students who have to do a dissertation in year 3 and (2) next year’s course. • Student self-assessment: “what I need to learn.” • The RSE scale has been adopted by ESRC Researcher Development Initiative.

17. Outcomes of teaching about Partnership & Interprofessional Practice Imogen Taylor, Clare Ockwell, Suzy Braye University of Sussex Note: Class-room based module does not include students from other professions.

18. Methodology • Outcomes of learning and teaching on BA & MA students’ attitudes • Stage 1 pre module teaching • Stage 2 post module teaching • [Stage 3 end of course] • Validated scales from University of West of England IPE programme • Communication and teamwork (CT) • Interprofessional learning (ILL) • Interprofessional interaction (II) • Interprofessional relationships (IR)

19. Mean ratings at start/end of module (BA)

20. Differences in attitudes by years ofpre-course experience

21. Next steps • Complete analysis of BA/MA T1 and T2 • Collect and analyse end programme BA/MA data (T3) • Compare with UWE IPE findings • Introduce new comparator programme (any offers?)

22. Outcomes of Teaching and Learning communication skills Juliet Koprowska University of York

23. Research question & methodology Are communication skills improved through learning within the university? Methodology: multiple-measure case study design. • Sample: year 1 BSc Social Work

24. Research methods • Self-efficacy scale • Recorded interview with service user-actor • Self-evaluation of interview • Service user-actor evaluation • Assessment of video-recording • Feedback from service users on placement • T1 = prior to teaching, T2 = 3 months later, after teaching, T3 = 9 months later, after first placement

27. Implications • Students may lose confidence as they acquire knowledge and understanding • Potential for formative assessment • Effect of placement needs exploration • Qualitative discussion with students and actors would deepen understanding • Should tools become regular measures to increase participation in research, with retrospective permission?

28. Communication Skills with Children & Young People Michelle LeFevre University of Sussex How does a Programme contribute to students’ development of confidence and competence in communicating with children and young people?

29. What and How are these learned? • ‘Skilled/effective communication’ is not just techniques and micro skills. Also underpinning knowledge, values and ethical commitments, personal qualities and emotional capabilities. • Learned throughout the whole programme not just in focused skills teaching – and also through other personal and professional experiences.

30. MethodologyProspective Evaluation – 4 stages • Questionnaire • Students’ personal characteristics: Have particular kinds of student learned most/least? • Measuring self-confidence in communication with children at different stages. • Which aspects of the programme facilitated your confidence and skills? • Case Vignette tool • Students demonstrate knowledge of planning, implementing, reviewing and reflecting on communication with children. • More objective measure (though analysis is subjective?)

31. Key Findings • Students’ confidence has increased. • Direct practice in placement the most important, followed by Child Development • All aspects of focused skills teaching were perceived as helpful to learning • Knowledge of aspects of communication skills increased (often significantly) - but not in every domain. • Still to analyse relation of characteristics to learning and mapping individual students through programme.

32. Learning about working in partnership with service users Roxana Anghel & Joanna Fox Anglia Ruskin University Exploring the use of Concept Mapping

33. Method – Concept Mapping (CM) “Schematic representation of an individual’s understanding of a knowledge domain in the form of concepts meaningfully linked in propositions.” • Design • CM and questionnaire at T1 (induction week BA); T2 (end year 1); T3 (end year 2) • CM task, unstructured - ‘Working in Partnership with Service Users’ • Questionnaire – most impacting learning opportunities. • Analysis of CM • Generate scores based on validated concept-links • Raters included service user and researcher.

34. Concept Map at start of programme

35. Concept map at end of Year 1

36. Concept Map at end Year 2

37. Map Scores Findings – “Working in Partnership”

38. Sources of Learning • At T1: Ethics and Values; observational practice; Poverty, Social Exclusion and Social Work, and academic reading. • At T2:Social Work with Children and Families, Social Work with Adults, practice placement; Principles and Skills of Social Work; academic reading; and discussing with colleagues

39. Module A – Map scores correlated with marks • Module A (Year 3 BA, 13 pairs) – 46% T2 maps increased content and quality Map Scores CM used on two other modules Students • Module B (Year 1 MA, 11 pairs) – 72% T2 maps increased content and quality • Module B – Map scores and marks did not correlate Map Scores Students

40. Discussion • Small numbers, so the value of CM as summative method in assessing outcomes needs to be explored further • Success depends on the clarity of instructions and purpose, familiarity and time available. • Uses: self-assessment, visual learning aid, formative assessment (identify gaps), as well as assessing outcomes. • Being firmly based on “meaningful” learning CM can be applied to many area of social work.

41. Measuring the Acquisition of Competency Sharon Vitali Oxford Brookes University

42. Methodology • Cohort Sample BSc Social Work • (n=38 at beginning n=34 at present = 89%) • Longitudinal - Baseline to Graduation • Repeated MeasuresFixed Outcomes (National Occupational Standards Units) RICET • Multiple Methods

43. Year One - Semester One  • Baseline • December RICET   RICET PERSONAL LEARNING PLAN

44. Consider your current state of competence (knowledge/skills) and indicate a score (from the guide below) for each of the learning outcomes listed. 0 = Can not produce any evidence of competence. 1 = Understands the learning outcome, but can produce only limited or no evidence of appropriate attempts to put it into practice. Much more knowledge/practice needed. 2 = Understands, and can offer evidence of tentative attempts to integrate into current knowledge/skill base. 3 = Demonstrates competence with some regularity. 4 =Advanced understanding and demonstrating adequate level of integration of knowledge, skills, and appropriate application. 5 = Clearly understands and demonstrates consistent and appropriate application of knowledge and skills in practice.

45. Academic Average = 61.4% Video Scores SU=1.8 ER=2.2

46. Year Three - Semester Two May (70 DAY PLO) RICET PRACTICE & PORTFOLIO TUTOR EVALUATION CRITICAL CAREER REVIEW TUTOR EVALUATION PRACTICE & PORTFOLIO P/T EVALUATION RICET

47. PA End Place2 ST End Place2 PA Mid Place2 ST Mid Place 2 PA End Place1 ST End Place1 PA Mid Place1 ST Mid Place1 ST End Yr1 S2 Tutor End Yr1S1 ST End Yr1 S1 ST Baseline

48. Types of Data NOW end

49. Overview of the findings from projects increases in: • Attitudes to race and racism - but not to self-confidence in challenging racism. • Self-efficacy following learning (e.g. research skills) Do these correlate with marks? • Conceptual understanding (C Maps, communication vignettes). • Ratings of Competence (RICET)

50. But: • No measured increase in Communication skills • No overall increases in attitudes to partnership working • Measures do not necessarily correlate with marks. And we need to strengthen research designs, especially comparative studies.