Design and Analysis of Single-case Research

1. IES Project Director’s Meeting June 2010 Rob Horner University of Oregon Design and Analysis of Single-case Research

2. Congratulations • Tom Kratochwill • Enhancing the scientific credibility of single-case design intervention research: Randomization to the rescue. • Larry Hedges • A d-estimator for single-case designs.

3. Implications • Contribution of single-case research for documenting evidence-based practices in education. • Special relevance for low-incidence populations • Useful pilot model prior to RCTs • Effective strategy for assessing fine-grained effects.

4. For Single-case Researchers • Adopting a multi-tiered model of analysis. • Does the design allow documentation of experimental control? • Three demonstrations of effect each at a different time • Do the data document a functional relation? • Level, trend, variability, immediacy, overlap, consistency • What is the size of the effect? • Small, medium, large • What is the social importance of the effect? • Subjective, contextual • What are the conceptual contributions of the effect?

5. For Single Case Researchers • Improving clarity with which visual analysis rules are defined, taught, used. • Reversal designs • Multiple baseline designs • Alternating treatment designs • Changing criterion designs • Designs employing multiple design elements Interpreting the whole data set for a study: Not just two phase comparisons

6. Intervention X

8. First Demonstration of Effect Third Demonstration of Effect Second Demonstration of Effect Comparison of actual against projected data (Analysis of Transition States versus Analysis of Steady States)

9. Implications for Single Case Researchers • Revise research questions: • from • Is there a functional relation between IV and DV? • to • Is there a functional relation between IV and DIMENSION of DV (level, trend, variability) • Implication for the β weights in an HLM analysis

10. Baseline Intervention Baseline Intervention

11. Implications for Single Case Researchers • Number of data points per phase • Fewer than 5 data points will create challenges • Randomization • Timing of phase introduction • Number of data points needed

12. Implications for the field • Single case research is an effective design method for documenting causal relations. • Single case designs are needed for us to understand what works, where, and how for low-incidence populations.

13. Implications for the field • Meta-analysis of single-case research is essential for broad adoption of the single case contributions. • Defining multiple design/statistical models for defining interpretable effect size measures (d) will be important for useful meta-analyses.

14. Multiple Options for Assessing Effect Size • d – estimator • Hierarchical Linear Modeling • Generalized Least Squares • Randomization • Percentage of Non-overlapping Data.

15. Summary: Using Single Case Designs • Understand the behavior of individuals • Understand change across time • Multi-tiered interpretation • Design • Visual Analysis • Statistical Analysis • Social and Conceptual Importance