From Evidence-based Practice to Practice-based Evidence: Behavior Analysis in Special Education

1. From Evidence-based Practice to Practice-based Evidence: Behavior Analysis in Special Education Ronnie Detrich Wing Institute

2. What is Evidence-based Practice? • EBP is a decision-making approach that places emphasis on evidence to: • guide decisions about which interventions to use. • evaluate the effects of an intervention. • Sackett et al (2000) defined evidence-based practice in medicine as: “the integration of best research evidence with clinical expertise, and patient values.”

3. What is Evidence-based Practice? • Ultimately, EBP is a consumer protection issue. • Assumes that evidence-based interventions are more likely to be effective than interventions that are not evidence-based. • By validating interventions as evidence-based there is the implication that there are standards for reviewing interventions. • Standards should be transparent.

4. The Scope of the Problem • Kazdin (2000) identified 550 named interventions for children and adolescents. • A very small number of these interventions have been empirically evaluated. • Of those that have been evaluated, the large majority of them are behavioral or cognitive-behavioral.

5. The Scope of the Problem • We have a great deal of evidence but much of that evidence is not being applied in practice settings. • Evidence-based interventions are less likely to be used than interventions for which there is no evidence or there is evidence about lack of impact (Kazdin, 2000). • In many instances practitioners are not aware of evidence-based interventions (Kratochwill, Albers & Shernoff, 2004). • There is very little guidance to parents and practitioners about how to decide among interventions.

6. Why the Increased Interest in Evidence-based Interventions? • No Child Left Behind (NCLB) requires that interventions used to improve educational performance are based on scientific research. • In NCLB there are over 100 references to scientific research. • Individuals with Disabilities Education Improvement Act [IDEIA] (2004) requires that interventions are scientifically based instructional practices.

7. Special Education and Evidence-based Interventions • Pre-service and professional development for all who work with students with disabilities to ensure such personnel have the skills and knowledge necessary to improve the academic achievement and functional performance of children with disabilities, including the use of scientifically based instructional practices, to the maximum extent possible.

8. Special Education and Evidence-based Interventions • Scientifically based early reading programs, positive behavioral interventions and supports, and early intervention services to reduce the need to label children as disabled in order to address the learning and behavioral needs of such children.

9. Special Education and Evidence-based Practice • The Individualized Education Program (IEP) shall include a statement of the special education and related services and supplementary aids and services, based on peer-reviewed research to the extent practicable,to be provided to the child, or on behalf of the child, and a statement of the program modifications or supports for school personnel that will be provided for the child.

10. Ethical Conduct and Evidence-based Interventions • Most national psychological and educational organizations have ethical standards requiring science-based practices to address problems. • American Psychological Association Ethical Standard 2.04: • Psychologists’ work is based on the established scientific and professional knowledge of the discipline.

11. Ethical Conduct and Evidence-based Practice • National Association of School Psychologists • Standard III F 4. • School psychology faculty members and clinical or field supervisors uphold recognized standards of the profession by providing training related to high quality, responsible, and research-based school psychology services.

12. Ethical Conduct and Evidence-based Practice • National Association of School Psychologists • Standard IV 4. • School psychologists use assessment techniques, counseling and therapy procedures, consultation techniques, and other direct and indirect service methods that the profession considers to be responsible, research-based practice.

13. Ethical Conduct and Evidence-based Practice • Behavior Analysis Certification Board • Standard 2.09a • The behavior analyst always has the responsibility to recommend scientifically supported, most effective treatment procedures. Effective treatment procedures have been validated as having both long-term and short-term benefits to clients and society. • Standard 2.09b • Clients have a right to effective treatment (i.e., based on the research literature and adapted to the individual client).

14. Controversies in Evidence-based Education • There is no consensus about what constitutes evidence. • The definition offered in NCLB permits both quantitative and qualitative evidence without specifying the types of questions that each approach best answers. • In this context, we are most often concerned with evidence that establishes a causal relation between an intervention and a class of social or academic behaviors. .

15. Controversies in Evidence-based Education • There is no agreed upon standard for either the quantity or the quality of evidence necessary to validate an intervention as being evidence-based. • Several organizations have established standards but there is limited agreement among them. • It is possible for an intervention to meet one standard but not a second.

16. Applied Behavior Analysis and Evidence • The research methods of single participant design are convincing demonstrations of causal relations. • Behavior analysis research has often been criticized for limited generalizability because of the small number of participants in a research study.

17. Applied Behavior Analysis and Evidence • Generally, the developed standards for validating an intervention as evidence-based have relegated single participant designs to a lower quality of evidence. • This is problematic because in some areas of educational research almost all of the evidence is based on these designs. • Autism • Developmental Disabilities

18. Applied Behavior Analysis and Evidence • There are no established standards within applied behavior analysis for validating interventions. • There is no single resource that decision-makers can turn to for guidance about the best intervention to use for a particular problem. • May lead decision makers to base decisions on criteria other than evidence.

19. Applied Behavior Analysis and Evidence • Some related fields have published standards: • Horner, et. al. (2005) provides guidelines for evaluating evidence established with single participant research. • Flay, et. al. (2005) standards of Society for Prevention Research. • National Standards Project is developing standards of evidence for interventions for individuals with autism. • Task Force on Evidence-based Interventions in School Psychology. • What Works Clearinghouse. • Only WWC has reviewed the literature in a specific area to provide guidance about effective interventions.

20. Implementation: Research to PracticeWhere good interventions go to die • Identifying evidence-based interventions is necessary but not sufficient to assure they will be implemented in practice settings. • It is also necessary to address complex issues associated with implementation (research to practice).

21. Four Concerns for Implementation • Dissemination of evidence-based interventions in a manner that is accessible and useful to decision-makers. • Publication in professional journals is not an effective dissemination strategy for decision makers. • Evidence about interventions are distributed across multiple journals. • Most practice level decision-makers are not adequately trained to evaluate primary source data. • Because of time constraints, reading professional journals is a low priority activity for many decision-makers.

22. Four Concerns for Implementation • Selection of evidence-based interventions. • Often variables unrelated to effectiveness influence selection of an intervention. • Cost • Expert opinion • Personal experience • Effort associated with systems change

23. Four Concerns for Implementation • Initial Implementation of new intervention • What is necessary to gain practitioner support? • Often inconsistent with previous training and experience. • What is required to train practitioners? • What is necessary to assure integrity of implementation? • Implementation without integrity is implementation of an unspecified intervention with unknown effects. • How can new intervention be adapted to meet local circumstances? • If changed too much it becomes a different intervention for which there are no data. • If not modified to meet local circumstances it may not be implemented at all.

24. Four Concerns for Implementation • Sustainability is the ultimate goal for successful interventions. • Definition of sustainable programs: • Maintains over time. • Maintains across generations of practitioners. • Supported with existing resources of system. • If effective interventions are not sustainable then the risk is great that alternative, ineffective programs will emerge. • The larger the scale of implementation the more complex all of the issues become and sustainability is more difficult to achieve.

25. Applied Behavior Analysis and Implementation • Service delivery in applied behavior analysis is a mediated model. • Behavior Analysts coach parents, teachers, or other mediators to implement an intervention plan. • Mediated model requires that behavior analysts address many of the issues of implementation for each project.

26. Progress MonitoringPractice-based Evidence • Even if an evidence-based intervention is implemented well, it is still necessary to systematically evaluate the impact. • Progress monitoring is practice-based evidence about the effects of an evidence-based practice. • It is difficult to predict which students will benefit from intervention prior to implementation. • No intervention will be effective for all students. • Progress monitoring is consistent with ethical guidelines.

27. Ethical Standards and Progress Monitoring • National Association of School Psychologists • Standard IV C 1b. • Decision-making related to assessment and subsequent interventions is primarily data-based. • Standard IV 6. • School psychologists develop interventions that are appropriate to the presenting problems and are consistent with the data collected. They modify or terminate the treatment plan when the data indicate the plan is not achieving the desired goals.

28. Ethical Standards and Progress Monitoring • Behavior Analysis Certification Board • Standard 4.04 • The behavior analyst collectsdata or asks the client, client-surrogate, or designated other to collect data needed to assess progress within the program. • Standard 4.05 • The behavior analyst modifies the program on thebasis of data.

29. Progress Monitoring and Special Education • Progress monitoring is fundamental to the IEP process. • Progress monitoring is at the heart of Response to Intervention (RTI). • All students are routinely and systematically monitored to determine if they are making adequate progress against established standards.

30. Progress Monitoring and Behavior Analysis • Data collection and review is the sine qua non of applied behavior analysis. • It is not applied behavior analysis if intervention data are not collected and reviewed to determine impact. • Behavior analysts rely on direct measures of behavior and its products to determine progress. • Words read correctly as a measure of reading ability. • Frequency of specific problem behaviors or adaptive behaviors.

31. Progress Monitoring and Behavior Analysis • Behavior Analysis best represents the relationship between evidence-based practices and practice-based evidence. • For many areas in special education there are a wide range of behavior analytic evidence-based interventions. • The fundamental emphasis on progress monitoring in behavior analysis makes it well suited for leading the movement to a comprehensive approach to evidence-based practices.