N.E.A.R. Narrative Evaluation Action Research Trainers: Jaime Timmerman Melanie Block

1. N.E.A.R.Narrative Evaluation Action ResearchTrainers:Jaime Timmerman & Melanie Block Western Region Health Centre

2. Format for today 1.00pm - Warm-up and introduction 1.15pm - What is NEAR? 1.50pm - What makes a good story? 2.05pm - Forming your own stories 2.45pm - Afternoon tea break 3.00pm - Tips and Resources 3.15pm - Support and assistance 3.30pm - Where to next? 3.45pm - Reporting session 4.00pm - End of workshop

3. Introduction Project history: WRHC & ISIS PC were pilot agencies Dr Yoland Wadsworth, Swinburne University of Technology Gai Wilson, Latrobe University Dr. Ani Wierenga, University of Melbourne DHS � Karen Goltz Draft manual produced containing case studies

4. Why this project? What is �Narrative�? What is �Evaluation�? What is �Action Research�?

5. Why this project? DHS reporting requirements Need to express qualitative data Quantitative data not enough Shows a richer more in depth view of what we do

6. What is narrative? A narrative is a story Narratives can be a spoken or written account of connected events in order of their happening. They can be in many forms such as an article, a letter, a short story etc Narratives provide rich comprehensive data. EXAMPLE After extensive research the decision was made to consult with the local elderly Vietnamese community to establish their health needs.� �We chose the Vietnamese community for this project due to the high population numbers and the low numbers accessing our services. We wanted to know answers to so many questions? How could we make our services more relevant to their needs? But the sheer number of Vietnamese people in our neighborhood was daunting, so we chose a smaller group to begin with. We began by talking to the local elderly Vietnamese community� EG 2IPC tours � 19 participants� �Nineteen Vietnamese community members have had tours of our centres in the Vietnamese language. Now they know where we are and what will happen when they walk in the door�. EXAMPLE After extensive research the decision was made to consult with the local elderly Vietnamese community to establish their health needs.� �We chose the Vietnamese community for this project due to the high population numbers and the low numbers accessing our services. We wanted to know answers to so many questions? How could we make our services more relevant to their needs? But the sheer number of Vietnamese people in our neighborhood was daunting, so we chose a smaller group to begin with. We began by talking to the local elderly Vietnamese community� EG 2IPC tours � 19 participants� �Nineteen Vietnamese community members have had tours of our centres in the Vietnamese language. Now they know where we are and what will happen when they walk in the door�.

7. Mug exercise We evlautate all the time, everyday. We weigh up options nd outcomes and make decision about what is �best�. We evlautate all the time, everyday. We weigh up options nd outcomes and make decision about what is �best�.

8. What is �Evaluation�? Evaluation is simply the process by which we judge the worth or value of something (Suchman 1967) Evaluation is the relationship between expectation and reality Synonyms: assessment, estimate, appraisal, valuation, estimation (program evaluation) A process of gathering and analyzing information for the purpose of determining whether a program is carrying out the activities that it had planned and the extent to which the program is achieving its stated objectives (through these activities). The evaluation is used as a tool to learn how the program is most effective and/or what modifications should be made to improve services Evidence is important because it provides: Accountability Allows us to make judgements shows effectiveness, worth or value Helps in improving practice Making a case for health promotion ( we are completing again illness treatment) (program evaluation) A process of gathering and analyzing information for the purpose of determining whether a program is carrying out the activities that it had planned and the extent to which the program is achieving its stated objectives (through these activities). The evaluation is used as a tool to learn how the program is most effective and/or what modifications should be made to improve services Evidence is important because it provides: Accountability Allows us to make judgements shows effectiveness, worth or value Helps in improving practice Making a case for health promotion ( we are completing again illness treatment)

9. What is �Evaluation�? It helps us to find out things like: How are we going? How did we go? Did we achieve what we wanted to? Is it worth doing again? Helps measure effectiveness, worth and value Have there been any negative outcomes? Who was influenced by the program? Melanie�s amazing race exampleMelanie�s amazing race example

10. What is evaluation? Evaluation gives us evidence Evidence provides: Allows us to make judgments Can show effectiveness, worth or value Helps in improving practice Making a case for health promotion (we are competing again illness treatment) Why evaluate? Good professional practice Accountability to management and funding bodies Duty of care to those we seek to serve Good intension are not good enough There are many evaluation methods

11. Evaluation in practice The Amazing Race example

12. What is action research? Action Evaluation Research Process map in kit ( 5min to read)

13. Action Research in practice Reflection: (see Evaluation plan in kit) As physio�s we considered this group to be successful however we did not why, how and for whom? Why did people come rain, hail or shine?

14. Action Research in practice Design: (refer to Safe & Strong Evaluation Plan in kit) The problems: What is Safe & Strong achieving for our clients? How can we improve the program?

15. Action Research in practice Design: Who is asking the questions? Who has the problem? Who is to benefit from the evaluation? What do we need to find out, from who? Who is to participate and when & where: Who or what is to be researched?

16. Action Research in practice Design: Who is the researcher? Who is it all for? Critical reference group Who is it �for�? Other stakeholders & their interests

17. Action Research in practice Fieldwork: Methods? Open & strategic questions Consider Peavey & Wadsworth Strategic questioning tool (see kit) What questions will you ask of who? What ethics need to be considered? Do not jump to conclusions until enough evidence and observations have been gathered

18. Action Research in practice Analysis, Conclusions & feedback: Identifying themes and creating new theories Are there any other explanations? What could or should happen next? What are the possibilities? Feedback: Check with critical reference group

19. Action Research in practice Planning: Realistic, practical & achievable recommendations What resources would we need to put this into practice?

20. What makes a good story? Brain storm: What features makes a good story? Examples: Instructional piece Hydrotherapy piece What�s the story? � style? Mystery, comedy? What is your story going to illuminate about the project or program? Encapsulate the basic story in an inviting title Who are you telling the story to? Who is the audience? Aorientate the story to them. Why are they interested in the story? Are they even interested? Use the story genre, language, examples and terminology and style appropriate for them to �best get it� Who is telling the story? Who � story is it? Perpectives? Form your voice? What�s the story? � style? Mystery, comedy? What is your story going to illuminate about the project or program? Encapsulate the basic story in an inviting title Who are you telling the story to? Who is the audience? Aorientate the story to them. Why are they interested in the story? Are they even interested? Use the story genre, language, examples and terminology and style appropriate for them to �best get it� Who is telling the story? Who � story is it? Perpectives? Form your voice?

21. Forming your own story Who are you telling the story to & why? Identify the stakeholders & their interests (both positive & negative) e.g. From Safe & Strong- Physio�s interest to improve quality of service & advocate for resources but will be challenging for us if client feedback is negative

22. Forming your own story Who is telling the story? Single or multiply voices? I, we, they Which level? See �NEAR: A map� in resource kit

23. Forming your own story Gaining material for your story. Focus group Interview Reflection Observation Keeping a Journal

24. Forming your own story Privacy and ethics; Clear understanding of you & participants of: Process of evaluation Scope of evaluation dissemination Purpose of evaluation- not tokenism Possible consequences of what you write Plain language statement Process of eval- ability to change name, review document & withdraw their part if they don�t agree with how they have been representedProcess of eval- ability to change name, review document & withdraw their part if they don�t agree with how they have been represented

25. Tips and ideas You need a story to tell Everyone has their own writing style; find it and practice it. Take time away from usual duties to write Peers to review your story (writing, reading drafts and discussing what worked). Willingness to receive feedback Ability to ask the questions �Why, Who, How, When�. Writing alone � or writing together Writing in bits/or in its entirety

26. Tips and ideas: Writing style A story � not dot points. Need to be �expressive� in your writing (invoke curiosity, what happened next?). Honesty & accuracy Use direct quotes or statements (and provide context for quotes) Use simple language (use instead of utilise). Definitely no jargon Colorful informative headings. Provides context for the paragraph/s. If using quotes/other peoples stories, ensure that it is true and accurate (check with them).

27. Tips and ideas: Editiing Raw� writing can actually be the most powerful and you can lose that by over editing. Letting go of some of the smaller story lines Having to condense and refine what is being said. Debrief, blame � then ask the questions why, how, when? Value in thinking twice, reflecting, muse for a couple of weeks to think about any extra information. Enjoy

28. Resources Manual reference details What makes a good narrative? (resource G from manual) Case studies Community Development in Health Resources Collection Writing a good plot outline Annotated Bibliography of narrative evaluation materials and sources

29. Resources Circular Action Research map Plain language statement examples Levels diagram

30. Where to next? Support and assistance: ongoing Length: 4-6 pages Timelines (see hand out in kit)

31. Activity for future intentions Evaluation plan in kit based on Action Evaluation Research Process model

32. Evaluating today! Your feedback