Linda A. Dimeff, Julie M. Skutch, Milton Z. Brown, Sharon Y. Manning, & Eric A. Woodcock

1. Evaluating the Efficacy of a DBT Online Training Course Using a Rigorous Performance Based Test Linda A. Dimeff, Julie M. Skutch, Milton Z. Brown, Sharon Y. Manning, & Eric A. Woodcock Introduction Hypotheses Results For over a decade, investigators have sought to address the dissemination problem: the fact most treatment providers are not trained in or use the myriad of empirically supported therapies now available (Weissman, et al, 2006). The primary aim of our research was to develop an online training course (OLT) to further the dissemination of Dialectical Behavior Therapy (DBT). The efficacy of the DBT online training course was compared to “training as usual”, namely instructor-led training (ILT) and treatment manuals (“text”), with respect to clinical application of course content. All three training modes were comprised of equivalent information on the same topic – Marsha Linehan’s Dialectical Behavior Therapy (DBT) Skills Training Manual for Borderline Personality Disorder (Linehan, 1993). • We hypothesized that all participants would show an increase in score on PBRP over time, regardless of condition. • Participants in the OLT condition would outperform participants in both Text and ILT conditions. Data (n = 126) were analyzed using Hierarchical Linear Modeling (PROC MIXED models in SAS 9.1.3) to conservatively handle missing data within a participant across time points (missing data at post n = 17, at f/u n = 27), account for clustering (“random”) effects due to participants’ being nested within agencies, and clustering of repeated measures within individuals. Predictors in the analyses were condition, time, and condition x time interaction. Results showed a significant main effect of time for all three conditions, but no significant difference between conditions over time for total score (see Table 1). Findings were nearly identical for global score. Analyses were repeated, taking into account training completion rate data (n = 118). OLT and text condition participants who completed ≥ 80% of the training (n = 97) scored higher on PBRP total score at post than those who completed < 80% (n = 21; OLT: t = -2.60, p = 0.01; text: t = -2.57, p = 0.01). Results were consistent for global score. There was no significant condition x time interaction for only those participants who completed ≥ 80% for either total or global score. Method We conducted a randomized controlled trial (RCT) where 150 treatment providers were randomly assigned to one of three active conditions: Text, OLT, and ILT. A structured performance based role play (PBRP) was developed and utilized as a primary outcome variable for the trial to assess clinical application of DBT skills in a simulated clinical context using trained actors as fictitious clients. The PBRP was conducted at baseline, post-training, and at 90 day follow up assessment. Three role play scenarios were developed to represent a typical client with Borderline Personality Disorder (BPD) and participants were randomly assigned to the order in which they received each role play. Participants were oriented to the client’s profile as well as the context for the clinical intervention – a telephone call to the therapist, initiated by the client for DBT skills coaching following a distressing event. Research participants were informed that they had up to 15 minutes to coach the client (played by a trained actor) in relevant DBT skills. All calls were digitally recorded and coded. A panel of three DBT experts devised and tested a coding instrument to measure competency in the application of DBT skills coaching through an iterative process. Coding resulted in a computed total score (points awarded for suggesting & elaborating specific skills and deducted for incorrect or antithetical use), and an overall global score (an anchored, subjective rating of overall quality of DBT). Single measure intraclass correlation coefficients were calculated to determine raters’ initial reliability scores. The correlation coefficients between raters were .88 for total score and 0.70 for global score (n=30). Discussion Hypotheses were partially supported in that all participants did improve PBRP scores across time. However, OLT participants did not outperform participants from the text and ILT conditions. It is possible that the instrument, newly developed for this project, was not robust enough to capture differences and should be refined. In addition, content knowledge of DBT skills and knowledge of how to apply them may well be more separate constructs than this measure was able to detect. It is also possible that all training modalities should be improved to better teach the application of DBT skills to a coaching phone call. Because this is a new measure of DBT Skills coaching competency we have no objective standard with which to compare mean scores or change over time. It is encouraging that participants did improve in the simulated clinical scenario to a level of “moderate DBT.” More targeted instruction would likely boost improvement even more. Moreover, it is reasonable to assume that practice in application of this therapy would lead to better proficiency as well. Online training by its very nature lends itself well to these types of improvements. Courses can be highly interactive, directly engaging the clinician, and opportunities for practicing clinical skill can be easily incorporated into this modality. Anchors for global score: 0 = poor DBT (egregious or fewer than 2 skills), 1 = minimal DBT (suggested at least 2 skills; minimal to moderate incorrect use), 2 = moderate DBT (2-3 skills suggested; minimal to moderate incorrect use), 3 = good DBT (at least 3 skills, at least one thorough, minimal incorrect use), 4 = great DBT (one change skill, one acceptance skill, minimal misuse), 5 = model DBT (thorough coaching in at least one change and one acceptance skill, absence of misuse). Table 1 Summary of results of HLM analyses of the effects of time and condition on PBRP total and global scores Source dfFp Total score Time (T) 2, 134 84.02 <0.0001 Condition (C) 2, 149 1.96 0.14 T x C 4, 160 1.10 0.36 Global Score Time (T) 2, 115 59.52 <0.0001 Condition (C) 2, 120 1.35 0.26 T x C 4, 136 0.60 0.67 This research was supported by grants awarded to Dr. Linda Dimeff from NIMH (2 R44 MH065790-02 and 5 R44 MH065790-03) and NIDA (2 R44 DA015615-02 and 5 R44 DA015615-03