Evidence-Based Practice

1. Evidence-Based Practice

2. Summary of Presentation • Evaluating Interventions • Recommendations for Parents • Fads

3. What Interventions Should We Use? • The Right to Effective Behavioral Treatment • ABA Task Force (1987) • Published in JABA (1988) • http://www.abainternational.org/ABA/statements/treatment.asp • Individuals who are recipients of treatment designed to change their behavior have the right to: • A therapeutic environment • Services whose overriding goal is personal welfare • Treatment by a competent behavior analyst • Programs that teach functional skills • Behavioral assessment and ongoing evaluation • The most effective treatment procedures available

4. The Most Effective Treatment Procedures Available • An individual is entitled to effective and scientifically validated treatment; • In turn, the behavior analyst has an obligation to use only those procedures demonstrated by research to be effective.

5. Behavior Analyst Certification Board (BACB) Guidelines for Responsible Conduct • Section 2.09 Treatment Efficacy • http://www.bacb.com/consum_frame.html • The behavior analyst always has the responsibility to recommend scientifically supported most effective treatment procedures. Effective treatment procedures have been validated as having both long-term and short-term benefits to clients and society. • Clients have a right to effective treatment (i.e., based on the research literature and adapted to the individual client). • Behavior analysts are responsible for review and appraisal of likely effects of all alternative treatments, including those provided by other disciplines and no intervention.

6. As Behavior Analysts, we… • Maintain a healthy skepticism • Differentiate opinions, beliefs, and speculations from facts • Don’t make claims without supporting objective data • Skepticism ≠ cynicism • Being open and being skeptical might seem contradictory, but they’re not. This is the way to think scientifically. • Some phenomena that seem outlandish are valid • Lucid dreaming, extraordinary feats of human memory and appropriate clinical uses of hypnosis (as opposed to the scientifically unsupported use of hypnosis for memory recovery). • “We must keep our minds open but not so open that our brains fall out”

7. Ghezzi, Williams, and Carr (1999) • Preface • “Those who fall in love with practice without science are like a sailor who enters a ship without a helm or compass, and who never can be certain whither he is going” (Leonardo da Vinci)

8. Green (1996) • Behavior analysts use a number of practices to increase the objectivity of evidence about treatment effects • Operational definitions (specific and observable descriptions of behavior) • Measurement procedures are clearly specified • Interobserver agreement data are collected • Multiple measures of the effects of tx can be obtained – for example, • Direct observation • Standardized tests • Parent ratings of behavior

9. How Do Parents Choose a Treatment for Their Child with Autism? • Recommendation by pediatrician or other doctor • School • Other parent • Internet, book • Do you think these sources reliably recommend interventions based on the objectivity of the evidence?

10. Fads in Autism Treatment • Metz, Mulick, and Butter (2005): Google search - autism and treatment = 65 distinct interventions sold as effective for treating autism • Telepathy, injection of sheep stem cells, thyme, swimming with dolphins • Fads in the media • In autism treatment, fads tend to be harmful • Waste time • Waste money • Falsely raise hopes and expectations • Distract from effective treatments • In some cases, harm children and families

11. Why Do Ineffective/Unproven Treatments Become Fads? (Vyse, 2005) • They’re presented as relatively easy and with immediate effects by people who appear warm, sincere, and attentive • Best treatment is deemed distasteful or is hard to get • Alternate treatments are supported by popular culture, “feel” right, seem to make sense • Professionals or other people recommend them • Most professionals are not trained how to evaluate treatments • Autism treatment is a commercial enterprise

12. Why Do Ineffective/Unproven Treatments Become Fads? (Metz et al., 2005) • Parents are in a vulnerable position • Doing SOMETHING feels better than doing nothing • Grieving process • Avoidance of guilt • Urgency • Qualities that make parents dedicated and enthusiastic make them vulnerable to accept claims without close scrutiny • Nature of Autism • Cause is unknown • Autism is mysterious – myth of the “hidden inner child”

13. Even Well-Meaning Professionals Use Ineffective/Unproven Interventions (Metz et al., 2005) • Example of facilitated communication • Rationale provided for the therapy may be logical or sound convincing • But many are based on flawed theories about the cause of autism • For the overwhelming majority of autism treatments, anecdotes and testimonials are the only supporting evidence • Almost none stand up to reasonably rigorous scientific evaluation • Many therapies for autism in widespread use today have been shown to be ineffective in scientific studies • Some have been shown to be harmful • Still others have not been subjected to any rigorous evaluations

14. Where can we refer parents? • Association for Science in Autism Treatment • www.Asatonline.org

15. Evidence-Based Practice • Evidenced-based interventions • Evidence-based practices • Empirically supported treatments • Best practices

16. Evidence-Based Practice (EBP)O’Donohue and Ferguson (2006) • “the integration of the best available research with clinical expertise in the context of patient characteristics, culture, and preferences” (APA Presidential Task Force on Evidence-Based Practice, 2006, p. 273) • http://www.apa.org/practice/ebp.html • Other similar terms • Empirically-validated therapies (EVT) • Empirically-supported therapies (EST)

17. Development of EBPO’Donohue and Ferguson (2006) • 20 years ago - Clinical practice guidelines developed in medicine to • Help standardize decision-making in treatment • Encourage use of empirically sound treatments • Improve quality of medical services and reduce errors • Early 90s – American Psychiatric Association developed their own clinical practice guidelines • Intended to assist psychiatrists in decision-making in treatment • Tended to recommend pharmacotherapy over non-pharm treatments • Mid 90s – American Psychological Association (APA) Task Force publishes their own clinical practice guidelines…

18. “The Chambless Criteria” • Division 12 of the American Psychological Association (APA) – Clinical Psychology • Established a Task Force (headed by Diane Chambless) to identify and promote empirically supported psychological treatments in 1993. • Chambless et al. (1998) – available at http://www.apa.org/divisions/div12/est/97report.pdf

21. Weaknesses of Chambless Criteria O’Donohue and Ferguson (2006) • Do not take clinical significance into account • Focus on efficacy, not effectiveness • Will the same results be obtained in natural settings? Is the treatment practical to implement in terms of staffing, funds, expertise? • Most studies on the EBP list do not include participants with comorbid disorders • Are biased toward group designs and inferential statistics • Only 2 “good” group design exps, but at least 9 single-case exps • Many single-case designs don’t compare 2 treatments

22. Individuals with Disabilities Education Improvement Act (IDEA) & No Child Left Behind (NCLB) • NCLB of 2001: interventions should be based on scientific research • IDEA of 2004: scientifically based institutional practices for those in spec eds • Services in a child’s IEP should be based on “peer reviewed research the extent practicable (IDEA, 2004)

23. Professional Organizations • APA • “Established scientific and professional knowledge of the discipline” • National Association of School Psychology (NASP) • Practice should be based on scitific research

24. Horner, Carr, Halle, McGee, Odom, and Wolery (2003) • Single-subject research documents a practice as evidence-based when… • The practice is operationally defined • The outcomes and context in which the practice is to be used are defined (target behaviors affected, setting, age, skills, diagnosis, implementer) • The practice is implemented with fidelity (tx integrity data) • The change is the DV is shown to result from the IV – experimental control! • Effects of the practice are replicated across a sufficient number of studies

25. Make a table:

27. Problems with Current Clinical Guidelines • Studies examined were before 1999 • Process used by each of the task forces were not made public • Not comprehensive

28. National Standards Project-examined empirical evidence supporting interventions that could be used in school setting for those younger than 22 • Exclusion criteria • Medical/complementary/alternative • Axis II outside of MR • Studies that could not be easily performed in schools • Qualitative analyses or published in non-peer reviewed journals

29. Scientific Merit Rating Scale • Rated from 0-5 • Research design • Measurement of the DV • Measurement of the IV • Participant selection • Generalization

30. Coding • Beneficial treatment effects reported • Single-case designs: • Strong, moderate weak • Functional relationship has been established & is replicated • Pont of comparison across conditions exists • Magnitude of change is consistent • Percentage eon non-overlapping points is impressive • No treatment effect reported • Adverse treatment effects reported

31. National Standards Project • http://www.nationalautismcenter.org/about/national.php

32. References • APA Presidential Task Force on Evidence-Based Practice. (2006). Evidence-based practice in psychology. American Psychologist, 61, 271-285. • Chambless, D.L., Baker, M., Baucom, D.H., Beutler, L.E., Calhoun, K.S., Crits-Christoph,P., et al. (1998). Update on empirically validated therapies, II. The Clinical Psychologist, 51, 3-16. • Ghezzi, P.M., Williams, W.L., & Carr J.E. (1999). Autism: Behavior analytic perspectives. Reno, NV: Context Press. • Green, G. (1996). Evaluating claims about treatments for autism. In C. Maurice, G. Green, & S.C. Luce (Eds.), Behavioral intervention for young children with autism. (pp. 15-28). Austin, TX: Pro-ed. • Metz, B., Mulick, J.A., & Butter, E.M. (2005). Autism: A lat-20th-century fad magnet. In J.W. Jacobson, R.M. Foxx, & J.A. Mulick (Eds.), Controversial therapies for developmental disabilities. (pp. 237-263). Mahwah, NJ: Lawrence Erlbaum. • O’Donohue, W., & Ferguson, K.E. (2006). Evidence-based practice in psychology and behavior analysis. The Behavior Analyst Today, 7, 335-350. • Vyse, S. (2005). Where do fads come from? In J.W. Jacobson, R.M. Foxx, & J.A. Mulick (Eds.), Controversial thearpies for developmental disabilities. (pp. 3-17). Mahwah, NJ: Lawrence Erlbaum.