Research-Practice Gap in Era of Evidence-Based Mental Health

1. Practice-based research on the Effectiveness of Psychotherapy and Psychotherapy Training: Research Framework and ProtocolsRobert ElliottUniversity of Strathclydefac0029@gmail.com

2. Research-Practice Gap in Era of Evidence-Based Mental Health • Numerous contemporary attempts to link research & practice in psychotherapy • Top-down solutions: • Empirically-supported treatments • Evidence-Based Practice/NICE Guidelines • Based on: • Randomized Clinical Trials research model • Therapist-as-research-consumer model • Results have been mixed

3. Research-Practice Integration as a Two-way, Dialectic Process • Success is more likely if we add a more integrative, bottom-up strategy • Building on Mental Health Services/ Therapy Effectiveness research paradigm • Existing RCT research makes space for grass-roots-based research in real world practice and training settings • =Practice-based Evidence (Lambert)

4. Practitioner Research Networks (PRNs) • USA: Pennsylvania (Ragusea, Borkovec, Castonguay) • UK: National Health Service CORE PRN (Barkham, Evans et al.)

5. Practice-Based Therapy Research in Training Sites • Training site research movement: USA, Europe • Research on psychotherapy process/outcome is essential for understanding and improving psychotherapy practice in all orientations • Being able to use and carry out research is an important aspect of therapist competence • Best way to learn about therapy research: • Do research during basic therapy training • Primary professional socialization process • Create habits that carry over into later practice

6. Principles of Practice-Based Therapy Research 1. Practical: Use inexpensive and easy-to-use instruments that can enhance therapy rather than interfere 2. Stakeholder-based: Actively involve therapists (and service users) in selection of research questions and methods 3. Focused: Include key elements (therapeutic alliance, client problem distress) 4. Incremental: Start simple with key elements and build (e.g., add theory-specific outcome measure) 5. Methodologically pluralist: Encourage variety of research methods (qualitative & quantitative; group & single-case) 6. Collaborative: Create research networks of training sites using similar, pan-theoretical instruments

7. Example of Practice-Based Research Initiative • International Project on the Effectiveness of Psychotherapy and Psychotherapy Training (IPEPPT) • Italian Initiative (June 2004) • General Goal: To improve psychotherapy and psychotherapy training in a broad range of theoretical approaches by encouraging systematic research in therapy training institutes and university-based training clinics.

8. IPEPPT: Current Status • Not a single study • “Project” = Promoting practice-based research in Europe, North American and elsewhere • Method development (instruments, framework) • Current active partners: • Scotland (Universities of Strathclyde, Abertay) • Belgium (KU Leuven & network) • Canada (St. Paul University & consortium of counselling training courses) • Italy (Institute for the Person-Centered Approach)

9. IPEPPT: Specific Objectives • 1. To formulate and promote use of a general framework for developing research protocols: • Not a “Core Battery” • Key aspects of therapy outcome and process, especially in training centers • Key aspects of therapy training outcome and process • 2. To facilitate the development of specific treatment and training outcome protocols for particular: • Therapy approaches (e.g., Systemic therapy) • Client populations (e.g., people living with schizophrenia) • Linguistic/national groups (e.g., Italy) • 3. To facilitate national/international collaborations

10. Framework for Practice-Based Research • Coordinated practice-based research requires a guiding conceptual framework for guiding what to measure and how to measure it • 3 dimensions: • (1) Research focus (2): Psychotherapy vs. Psychotherapy Training • (2) Research topic (3): Background, Process, Outcome • (3) Level of generality (2): General vs. Specific

11. Framework: Six Therapy Measurement Domains, with examples of key concepts

12. Dimension 2: “Star” Design General vs. Specific • Main body of the star = General outcome/ process/background protocol • Shared across orientations/client populations/ languages • Provides common metric • Star rays = Specialized protocols for different therapy approaches and different countries (Theory/Population/Language- Specific)

13. Dysfunctional Attitudes Self-Ideal Discrepancy CBT Experi- ential Target Problems Experiential Access Implicit Cognitive Biases Self-Esteem General problem severity Interpersonal/ relational issues Qualitative perceptions of change CCRT Change Relational Satisfaction Maturity of Defenses Family Environment Level of Object Relations Interpersonal Empathy Psycho- dynamic Family/ Couples “Star” Design for Sample Concepts within Therapy Outcome Domain for Studies of Four Different Therapies

14. Strategy for Selecting Instruments: Nested Priority Lists • Not a single “core battery” • Allow flexibility while encouraging consistency within & across approaches • Three Levels of Priorities: • Measurement domains are prioritized • Within each measurement domain, key concepts are ranked by approximate importance • For each concept, available instruments are also described (researchers prioritize)

15. (1) Research Priorities across Therapy Measurement Domains

16. (2) Example of Proposed Concept Priorities • General Therapy Outcome Domain: • (1) General problem severity (quantitative) • Give every 2 sessions to reduce data loss from drop-out • (2) Qualitative perceptions of change • (3) Interpersonal/relational functioning • (4) Individualized problems/goals • (5) Health care utilization/costs • (6) Quality of life/life satisfaction/well-being

17. (3) Researcher Prioritizes Common Measures of Key Concept: E.g., General Symptom Severity Instruments

18. Framework: Six Therapy Measurement Domains, with examples of key concepts

19. General Therapy Process Domain • Key concepts in possible recommended priority order: • (1) Therapeutic alliance • (2) Perceived helpful aspects of therapy • (3) Perceived session effectiveness • (4) Therapist and client response modes

21. Different Levels of Research Protocol are Possible • I. Minimum Protocol • II. Systematic Case Study Protocol Other Protocols: • III. Training Research Protocols • IV. Specific Research Protocols

22. I. A Recommended Minimum Protocol: Applications • Easy to use: Limited to one measure from each of the general therapy domains • Provides basic treatment monitoring for individuals & agencies • Trainers model in their practice • Students use in practicum-placement settings • Other versions are possible (e.g., different outcome or process measures)

23. I. A Recommended Minimum Protocol: Key Concepts • (1) General therapy outcome instrument • Client problem severity • Give at odd-numbered sessions (short form) • (2) General therapy process • Therapeutic alliance (use short from) • (3) General client/therapist background measure • Standard practice: • Client/ therapist demographics • Client diagnosis, presenting problems • Type of therapy

24. II. Systematic Case Study Protocol: Applications • Use for student case study requirements • Meets emerging standards for systematic single case research • New online journal: Pragmatic Case Studies in Psychotherapy (Rutgers University, Editor: Fishman)

25. II. Systematic Case Study Protocol: Elements • A. Therapy Outcome: • (1) Weekly/biweekly outcome measure • (2) At least one other quantitative outcome measure • (3) Qualitative outcome assessment (e.g., post-therapy interview) • B. Therapy Process • (1) Therapeutic alliance • (2) Detailed record of therapy (process notes and/or recordings) • (3) Qualitative perception of helpful aspects (post-session and/or post-therapy) • C. Client/therapist background • Client/ therapist demographics; client diagnosis, presenting problem; type of therapy • = Carried over from Minimum Protocol + = Added for systematic case study protocol

26. II. Systematic Case Study Protocol: Research Questions • (1) Did the client change substantially over the course of therapy? • (2) If the client changed, did therapy make a substantial contribution? • (3) What brought about the client’s changes?

27. II. Systematic Case Study Protocol: Emerging Evidence Standards • (1) Rich case record, including both quantitative & qualitative data • (2) Replication/convergence across methods & clients • (3) Critical examination of alternative views (e.g., Hermeneutic Single Case Efficacy Design, Elliott, 2002): • Non-change explanations (e.g., measurement error) • Non-therapy explanations (e.g., extra-therapy events) • (4) Narrative coherence • Narrative model of predisposing and process factors • Use for generalizing to other cases

28. Example: The Strathclyde Systematic Case Study Protocol -1 • A. Therapy Outcome: G: CORE-OM (@10 sessions) G: Personal Questionnaire (weekly) G: Change Interview (Perceived changes; @10 sessions) PC: Strathclyde Inventory (@ 10 sessions) G: General; PC: Person-Centred; PE: Process-Experiential

29. Example: The Strathclyde Systematic Case Study Protocol-2 • B. Therapy Process G: Working Alliance Inventory-12-R client, (after session 3, 5, 10 etc) G: Helpful Aspects of Therapy Form (weekly; Client) G: Change Interview: Helpful/hindering processes (@10 sessions) PC: Therapeutic Relationship Scale (WEG; Client; session 3, 5, 10 etc.) PC: Relational Depth Scale (@10 sessions) PC: Therapeutic Relationship Scale (Therapist; session 3, 5, 10 etc.) PE: Experiential Session Form (Therapist; weekly) • G: General; PC: Person-Centred; PE: Process-Experiential

30. Example: The Strathclyde Systematic Case Study Protocol-3 • C. Client/therapist background G: Client/ therapist demographics; client presenting problems, presenting problem; type of therapy G: Change Interview (Strengths & Limitations) G/PC: Pretest score outcome measures • G: General; PC: Person-Centrered; PE: Process-Experiential

31. III. Training Research Protocols: Issues • Outcomes of therapy training not well understood • Difficulties: • Lack of agreed-upon measures of therapist functioning and skill • Must measure therapist change longitudinally over one or more years of training • Possible applications: • Use research to improve training • Meet requirements of accrediting and funding agencies

32. III. General Training Research Protocols: Promising Concepts • Trainee psychological functioning (e.g., clinical distress, level of functioning) • General therapist facilitative interpersonal skills (e.g., coping with common difficulties) • Quality of therapist professional involvement and growth (e.g., Orlinsky; Collaborative Research Network [CRN]) • Qualitative perceptions of effects and important aspects of training (e.g., Trainee Change Interview) • Change in therapist self concept (e.g., Scilligo, SASB Introject scales)

33. Example: Strathclyde Diploma Course Training Evaluation Protocol • A. Training Outcome: (1a) CORE-OM (general distress) (1b) Strathclyde Inventory (Person-centred outcome measure: Congruence/incongruence) (1c) CRN Process Form (Healing Involvement; Experienced Professonal Growth) (2) Trainee Change Interview (changes) (3) Client change (CORE-OM), therapeutic alliance • B. Training Process (1) CRN Process Form (helpful, hindering factors) (2) Trainee Change Interview (helpful, hindering processes) • C. Trainee background • CRN Trainee Background Form

34. IV. Specific Protocols • = Star rays • Applications: For specific theoretical approaches, client populations, or language groups • Requires working committee for each group • Identify relevant therapy outcomes, processes, background variables (or training outcomes) • Do protocol and measure development research • Establish virtual communities for exchanging ideas

35. IV. Strathclyde Person-Centred/ Experiential Therapy for Social Anxiety Study • Additional instruments (beyond Systematic Case Study Protocol): • Outcome: • SA: Social Phobia Inventory • PE: Self-Relationship Scale • G: Inventory of Interpersonal Problems • G: Health Utilization Questionnaire • Background: • SCID-IV (Research Edition) • Personality Diagnostic Questionnaire-4

36. Promising New Therapy Research Methods Make this Work Possible • Systematic qualitative research methods • Interpretive single case designs (Fishman, Elliott) • Using early outcome to identify & repair problems (Lambert: Signal alarm methods) • New, powerful psychometric methods (Rasch analysis/Item Response Theory) • Online resources: • Virtual communities (e.g., Community Zero; Google Groups) • Client tracking and scoring (e.g., Google Documents; Survey Monkey)

37. Invitation to Dialogue - 1 • 1. Provide comments and suggestions on the framework & concepts presented here: fac0029@gmail.com • 2. Form or join online discussion groups or virtual communities • Closed sites; must apply for membership • General info: www.ipeppt.net • Example: www.communityzero.com/pcepirp • 3. Begin implementing the minimum protocol design with your own clients and in your own training setting.

38. Invitation to Dialogue - 2 • 4. Convert traditional case presentation training requirements into systematic case study exercises • 5. Help with translations of key research instruments • 6. Contribute to psychometric research: • Improve existing instruments • Equate different instruments for same concepts • Collaborate with groups with similar interests to generate data for pooling.

39. Email: fac0029@gmail.comBlog: pe-eft.blogspot.com