Towards measuring learning experiences as outcomes of psychosocial Intervention

1. Towards measuring learning experiences as outcomes of psychosocial Intervention Willem Melief

2. Why the development of the learning concept? AThe intention to improve the scientific base of the outcome measurement instrument Sater B The consistent finding of learning experiences as outcomes in the application of Sater

3. What is Sater ? A standardized system to measure Benefits of Social Work Intervention Experienced by Clients Developed by Verwey-Jonker Institute: Willem Melief Meta Flikweert Norbert Broenink

4. Developed on Requests of stakeholders Regular requests for research about outcomes of social work client experiences client satisfaction By stakeholders Agencies for Social Work Representatives of clients Financiers and government

5. Characteristics of SATER • The use of client problem definitions • Client experiences linked to problem categories and intervention aspects • Clients enter answers in computer questionnaire for optimal privacy • Standard table and graph output produced by a few standard SPSS command files • Standard report format

6. Measurement of ‘Client Experiences’ Two elements 1Experienced benefits in relation to the experienced problems Benefits  results, outcomes 2 Satisfaction about personal treatment and treatment process .

7. Sater Data Clients of 6 agencies 5 in a pilot; 1 in a further application Database of appr. 1275 client interviews 6 agency reports

8. Benefit categories Problems partly or totally solved = Prob. Sol. Learned to accept and live with unchangeable conditions = Acceptance Learned to handle problems = Handling Experienced support and understanding = Sup/underst

9. Experienced Benefits in %; N = 250 (Gron) 1025 (Pilot)

10. Problem solving Learning Learning to handle problems including solving them Learning to accept and live with Unchangeable reality 33% 50+ % 20 + % Change oriented benefit categories

11. Arguments in favour of the further development of learning The frequent occurrence in benefits reported by clients Connection to social work methodology Perspective of lasting, repeatable effects of intervention Connections to neighbouring fields psychotherapy psycho-education

12. Questions to be answered about learning effects 1What are the learning effects that correspond to the sort of learning reported by the clients in our research? 2 How can we measure these learning effects? .

13. Remembering historical dates Remembering the periodic table of chemical elements Acquiring skill in mental arithmetic Acquiring skill in writing correct language Mastering the art piano playing Becoming a accomplished football player Acquiring skills in controlling ones violent mental impulses Finding out how to relate to other people Acquiring skills in solving (specific) problems Acquiring skills in staying out of (specific kinds of) trouble Kinds of learning

14. Psychosocial learning Knowledge elements Behavioral skills Repertoire of Learned Elements At disposition of person Attitudinal elements Thought procedures Selection from repertoireof elements matching the Psychosocial situation Adequate handling of the Psychosocial situation

15. Causes for inadequate handling of the situation • Missing elements in the repertoire • Shortcomings of elements in the repertoire: • Lack of information • Insufficient skills • Insufficient training in practice • Shortcomings in the ability to assess the situation, • Shortcomings in the control and co-ordination of the use of elements from the repertoire. • Factors blocking adequate use of elements: • Overwhelming emotions (fears, anger, depression, etc) • Serious psychopathology • Serious shortcomings in elementary conditions • Serious uncontrollable behaviour of other people

16. Main helping strategies • Removing the blockades first • Shortcomings in elementary conditions • Overwhelming emotions • etc to free the road for: • Teaching clients to accomplish lasting results: • necessary information • skills • ability to assess the situation • ability to apply the right element from his repertoire

17. Literature Search • Social Work Literature • Very little direct or systematic treatment of learning effects • Description of intervention outcomes that can be interpreted as learning for example behaviour modification directed treatment • A Dutch approach of learning as model for emanicipatory social work and empowerment • Social Science Literature : • The work of the Dutch psychologist and scholar A.D. De Groot, who intoduced the concept of fundamental learning experiences.

18. Traditional measurement of learning effects • In traditional educational research Learning effects = behavioural effects • Acquired knowledge • Skills • Attitudes • This approach is very valuable !!, but

19. Coverage problem but is not enough The limitation to only measuring behavioural effects causes a Coverage problem = • subjective fundamental learning experiences are not measured (covered)

20. Fundamental Learning Experiences • Conscious experience of person that: • He has learned what behaviours (skills, attitudes) he has at his disposition = ready to be used • He has learned when certain behaviour is adequate and when not • He has learned to flexibly use available behaviours adapted to new situations

21. Fundamental Learning Experiences • Have as much to do with things (knowledge behaviour) that are always as they are and reactions that always have to be the same: as with • Exceptions, things that are (sometimes) different and reactions that sometimes have to be different.

23. Relevance for social work • Terminology and content are very similar to our own rudimentary way of thinking. • Problems of clients relate to shortages in conscious application of behavioural elements due to: • lack of insight in what one has at one’s disposition • lack of skill to assess the situation and select the right reaction to it.

24. Preliminary conclusions • Measuring learning experiences by clients are important; • Provides core insights that are missing in current outcome research • This information cannot be measured in another way • There is a promising connection to theoretical thinking • Antagonism in the research and professional world is a serious obstacle

25. Opposing positions Pro objective measurement : -What is not objectively measurable does not count (is irrelevant) -Subjective experiences of clients are not to be trusted (have no validity) versus Against objective measurement -reduces the uniqueness of the client -subjective data are too important to be discarded -mostly strong preference for qualitative research

26. Supplementary Model • Quantitative outcome measurement using objective measurement = OK!! • Base of evidence base methods But should be supplemented by: • Quantitative measurement of subjective experiences, which = also OK • Learning experiences • Satisfaction • Qualitative research = certainly also OK , very usefull Social Work Literature • Very little direct or systematic treatment of learning effects • Description of intervention outcomes that can be interpreted as learning for example behaviour modification directed treatment • A Dutch approach of learning as model for emanicipatory social work and empowerment • Social Science Literature : • The work of the Dutch psychologist and scholar A.D. De Groot, who intoduced the concept of fundamental learning experiences.

27. The future • Expand the theoretical base for measuring learning experiences • On preference including contacts with scholars I missed in other countries. • Convert the very crude part about learning experiences in the Sater system to a more sophisticated measuring tool • To correlate experience outcomes with outcomes of objective measurement , very usefull Social Work Literature • Very little direct or systematic treatment of learning effects • Description of intervention outcomes that can be interpreted as learning for example behaviour modification directed treatment • A Dutch approach of learning as model for emanicipatory social work and empowerment • Social Science Literature : • The work of the Dutch psychologist and scholar A.D. De Groot, who intoduced the concept of fundamental learning experiences.