1 / 24

Qualitative Evaluation Workshop

Qualitative Evaluation Workshop. Basic Overview & Case Study Discussion Kim Koester AETC NEC UCSF. Workshop Overview. Review the concept, methods and uses of qualitative evaluation Discuss case study of an upcoming NEC evaluation

albert
Télécharger la présentation

Qualitative Evaluation Workshop

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Qualitative Evaluation Workshop Basic Overview & Case Study Discussion Kim Koester AETC NEC UCSF

  2. Workshop Overview • Review the concept, methods and uses of qualitative evaluation • Discuss case study of an upcoming NEC evaluation • Brainstorm opportunities to apply qualitative evaluation in your location

  3. Assumptions and Realities of Qualitative Evaluation What comes to mind?

  4. What is Qualitative Evaluation? [Qualitative research] Tries to picture the empirical world as it actually exists to participants, rather than how the researcher imagines it to be. (Filstead)

  5. Qualitative researchers empathize and identify with the people they study in order to understand how those people see things. Taylor and Bogdan

  6. Inductive Small n Purposeful sample Open-ended questions, geared towards discovery Seeking holistic understanding Not generalizable Investigates how or why Deductive Large n Random sample Closed-ended questions, predetermined measures Individual variables Generalizable Investigates how many when, where Comparing Qualitative & Quantitative Methods

  7. The Benefits of Qualitative Evaluation • Provides context • Provides nuance • Provides meaning • Provides richness • Provides explanatory power

  8. When to use qualitative evaluation? • Needs assessment • Assess whether trainees are applying knowledge • Assess barriers to applying training content • Assess trainees clinical environment • Quality assurance – staying on track

  9. Qualitative Methods • In-depth interviews • Observations • Focus groups • Content analysis

  10. Tools of Qualitative Research

  11. Qualitative Analysis: Otherwise known as “Transforming Data into Findings” -Patton • Driven by goals and aims of the evaluation • Describe clinic environment • Elucidate training dynamics • Identify patterns across interviews • Requires time to read & re-read data multiple times

  12. Challenges of Qualitative Evaluation • Resource intensive • Skilled evaluation staff needed • Adequate time to carry out evaluation & analysis • May be costly to carry out evaluation

  13. Examples of AETC Qualitative Evaluation • MP – Educating Healthcare Providers: How They Learn From and Apply Instruction • SEATEC – Evaluating Intensive On-Site Clinical Consultation • PAETC –Evaluation of the Pacific Island Jurisdiction MAI Project

  14. PART II: Case Study Understanding the Barriers and Facilitators to Applying the Content of AETC Training

  15. Background: Expanding Evaluation Domains Current AETC evaluation domains: • Assessing trainee reaction • Assessing trainee learning • Assessing behavior change • Assessing results of practice changes for patients New direction: • Assessing application of training materials

  16. Goals of NEC Evaluation • Primary: To qualitatively assess the barriers and facilitators to implementing and sustaining new knowledge and skills in order to design a quantitative instrument to evaluate the outcomes of training on provider practice and behavior. • Secondary: To check with each regional AETC and the NMAETC to assess the specific needs regarding outcome evaluation and evaluation of application of training content.

  17. Evaluation Questions • Are AETCs evaluating trainees’ application of training knowledge? If so, how? • If the trainee made changes based upon training, what were the barriers and facilitators to this process? If no changes were made, why not?

  18. Methods • In-depth interviews via Telephone • Two person on evaluation team – one to ask question, one to take notes

  19. Exercise • Sample: Who should we talk to? How many ppl will we need to talk to? • Protocol: How shall we proceed? What is our timeline? • Interview guide: What shall we ask them? • Pilot: Do we need to pilot the guide? • Analysis: How shall we approach the data? • Dissemination: How shall we disseminate?

  20. Parting Advice • Be collaborative. • Design your evaluation and training simultaneously. If not possible, work with your trainer to design evaluation that is feasible. • Be realistic – create an evaluation that can be carried out without undue burden.

  21. PART III: Applying Concepts Locally

  22. Resources • AETC National Evaluation Center http://www.ucsf.edu/aetcnec/index.html • Your AETC colleagues! • “Qualitative Research & Evaluation Methods” Michael Quinn Patton • Evaluation Journals: Qualitative Health Research & American Journal of Evaluation

More Related