Single Subject Research and Evidence-based Interventions: Are SSDs Really the Ugly Stepchild?

1. Single Subject Research and Evidence-based Interventions: Are SSDs Really the Ugly Stepchild? Ronnie Detrich Randy Keyworth Jack States Wing Institute

2. The Problem • Standards of evidence are necessary to identify evidence-based interventions. • Allow us to evaluate the strength of evidence across studies for a particular intervention. • Single subject designs have not always been accepted as a legitimate means for demonstrating the impact of an intervention. • What Works Clearinghouse has no standards for single subject designs.

3. The Problem • In many sub-disciplines within education and psychology single subject designs have been primary method for identifying effective interventions. • Developmental disabilities • Autism • Severe behavior problems • If single subject designs are not accorded scientific status then many effective interventions will not be validated as evidence-based interventions.

4. Characteristics of Single Subject Design • The purpose of SSDs is to demonstrate a functional relation between an independent and dependent variable. • Intense analysis of a few subjects demonstrates the functional relation. • Reliance on visual inspection good for identifying variables that have “whopping” effect. • Identifies socially significant effects. • Well designed studies control for threats to internal validity. • Internal validity: the degree to which alternative explanations for the obtained effects have been controlled for through the experimental design.

5. Characteristics of Single Subject Design • Demonstrates the robustness or generality of an independent variable through direct and systematic replication. • Direct replication: exposing the same or different subjects to exactly the same experimental arrangement that resulted in identifying a functional variable. • Reversal designs both within and across subjects • Multiple baselines across subjects • Systematic replication: varying some feature of the original experimental arrangement. • Different subject characteristics, different settings, different responses, different “doses” of the functional variable.

6. Generality and External Validity • Generality and External Validity are related but distinct concerns. • Generality describes the boundary conditions of a functional relation. • Under what conditions does the functional relation “break down”? • External validity refers to degree to which the results of a research study can be extended to other populations, settings, and conditions. • Degree of external validity is always contextual. Depends on the similarity between research and intervention conditions. • Answers actuarial questions that concern program administrators and policy makers- “how big a bang will I get and what is the probability of impact”?

7. Generality and External Validity • Single subject designs are most often criticized because of issues related to external validity. • In large part behavior analysts have not given much consideration to subject characteristics. • Behavior analysts have been more concerned with establishing the robustness of a few variables (reinforcement, stimulus control). • Body of knowledge is established through direct and systematic replication. • As we move from the study of single variables and basic behavioral processes to multi-component packages the distinction between generality and external validity becomes more confused.

8. Benefits of Single Subject Design • A rigorous methodology for identifying functional variables. • Allows scientist to see pattern of action of the variable of interest: • Can make informed statements about: • Acquisition • Maintenance • Generalization

9. Benefits of Single Subject Design • It is possible to study low incidence populations and behaviors. • Cost-Effective relative to group designs. • Can evaluate intervention before subjecting to large scale studies. • Close continuous contact with the data allow for great flexibility. • Research can be completed by scientist-practitioner in practice settings. • Can easily test clinical hypothesis. • Best method for progress monitoring in applied settings.

10. Limitations of Single Subject Design • Does not answer “actuarial” questions related to external validity very well. • Was not intended to answer those questions. • Reliance on visual inspection may result in unreliable interpretation. • There are no established standards for visually evaluating data. • Several researchers have criticized relying on visual inspection as means of interpretation (DeProspero & Cohen,1979) because of relatively low agreement between observers.

11. Limitations of Single Subject Design • Methods for aggregating results across studies have not been established. • Meta-analysis approaches may be useful. • This is very important for validating interventions as evidence-based. • Practitioners and decision-makers do not have time nor access to all of the primary source data. • Standards for validating interventions as evidence-based with SSDs are just emerging. • No consensus among these standards.

12. Examples of Standards for Single Subject Designs

13. Are SSDs the Ugly Stepchild? • Should not be • As long as they are used to identify functional variables. • But not everyone agrees: • Some excellent texts on group designs poorly describe SSDs. • We have work to do.

14. Recommendations • Develop appropriate meta-analysis methods for single subject research. • Develop standards for visual inspection. • Complex, very politically sensitive task. • Work with national organizations such as What Works Clearinghouse to assure that single subject research given equal status to group designs.