بررسی بنگ برای آموزش: کارآزمایی تصادفی تحت کنترل با مقایسه سه روش کلینیکی آموزشی در دو استراتژی اصلی رفتار درمانی دیالکتیکی

The present study examined the efficacy of online training (OLT), instructor-led training (ILT), and a treatment manual (TM) in training mental health clinicians in two core strategies of Dialectical Behavior Therapy (DBT): chain analysis and validation. A randomized controlled trial compared OLT, ILT, and TM among clinicians naïve to DBT (N = 172) who were assessed at baseline, post-training, and 30, 60, and 90 days following training. Primary outcomes included satisfaction, self-efficacy, motivation, knowledge, clinical proficiency, and clinical use. Overall, ILT outperformed OLT and TM in satisfaction, self-efficacy, and motivation, whereas OLT was the most effective method for increasing knowledge. The conditions did not differ in observer-rated clinical proficiency or self-reported clinical use, which both increased to moderate levels after training. In addition, ILT was particularly effective at improving motivation to use chain analysis, whereas OLT was particularly effective at increasing knowledge of validation strategies. These findings suggest that these types of brief, didactic trainings may be effective methods of increasing knowledge of new treatment strategies, but may not be sufficient to enable clinicians to achieve a high level of clinical use or proficiency. Additional research examining the possible advantages of matching training methods to types of treatment strategies may help to determine a tailored, more effective approach to training clinicians in empirically supported treatments.

.

.Results Sample Description No significant differences were found between conditions on any demographic variable (see Table 1). The modal study participant was a Caucasian female approximately 40 years of age with a master's degree and 6.5 years of clinical experience who was practicing in an outpatient community mental health clinic. Table 1. Sample Characteristics Characteristics OLT (n = 55) ILT (n = 55) TM (n = 62) Full Sample (N = 172) Between-Condition Comparisons Age M (SD) 42.0 (12.6) 39.4 (11.7) 38.2 (11.5) 39.8 (11.9) F (2, 215) = 1.5, p = .22 Gender Female 76.4% 67.3% 83.9% 76.2% χ2 (2) = 4.4, Male 23.6% 32.7% 16.1% 23.8% p = .11 Ethnicity Caucasian 76.4% 78.2% 80.6% 78.5% χ2 (10) = 6.8, Native American 1.8% 3.6% 0.0% 1.7% p = .74 African American 1.8% 7.3% 3.2% 4.1% Asian American 7.3% 3.6% 8.1% 6.4% Hispanic 5.5% 1.8% 3.2% 3.5% Other 7.3% 5.5% 4.8% 5.8% Education High School 5.5% 5.5% 3.3% 4.7% χ2 (10) = 5.6, AA/AS 10.9% 7.3% 1.6% 6.4% p = .85 BA/BS 27.3% 32.7% 37.7% 32.7% MA 47.3% 43.6% 47.5% 46.2% MD/PhD 5.5% 7.3% 6.6% 6.4% Other 3.6% 3.6% 3.3% 3.5% Profession χ2 (16) = 15.9, Psychologist 0.0% 5.5% 4.8% 3.5% p = .46 Psychiatrist 0.0% 1.8% 0.0% 0.6% Psychiatric Nurse Practitioner 1.8% 1.8% 0.0% 1.2% Chemical Dependency Counselor 25.5% 27.3% 12.9% 21.5% Social Worker (MSW) 16.4% 16.4% 16.1% 16.3% Mental Health Counselor (MA/MS/MFT) 21.8% 21.8% 19.4% 20.9% Mental Health Counselor (BA/BS) 3.6% 1.8% 4.8% 3.5% Student 3.6% 9.1% 11.3% 8.1% Other 27.3% 14.5% 30.6% 24.4% Work Setting χ2 (14) = 13.7, Private Practice 0.0% 1.8% 4.8% 2.3% p = .47 Outpatient CMHC 45.5% 41.8% 41.9% 43.0% Residential/Day Treatment Facility 10.9% 7.3% 3.2% 7.0% Drug Treatment Program 7.3% 7.3% 8.1% 7.6% Inpatient 7.3% 9.1% 8.1% 8.1% Corrections Facility 3.6% 0.0% 9.7% 4.7% School/Training 1.8% 3.6% 4.8% 3.5% Other 23.6% 29.1% 19.4% 23.8% Experience M (SD) Months worked as a treatment provider 76.7 (78.6) 77.6 (80.2) 83.4 (99.9) 79.4 (86.7) F (2, 745) = 0.1, p = .91 Prior DBT Experience None/minimal 84.6% 88.9% 83.9% 85.3% χ2 (2) = 0.6, Some/moderate 15.4% 11.1% 16.1% 14.7% p = .72 Note. Some categories do not total 100.0% because of rounding. Prior DBT experience is defined as: none/minimal (e.g., attended an in-service training, grand rounds, or lecture) or some/moderate (e.g., read part of the primary DBT manuals, attended a 1- or 2-day workshop). OLT = Online training. ILT = Instructor-led training. TM = Treatment manual. Table options Training Completion and Barriers Encountered Training Completion A significant Condition effect was found for percentage of training completed at the posttraining assessment, F (2, 142) = 3.95, p < .05. Participants in the ILT condition completed more of their training than those in the TM condition, t(142) = 2.78, p < .01. In general, participants had completed a high percentage of their training by the posttraining assessment: TM = 89%, OLT = 92%, and ILT = 98% complete. Barriers The three conditions did not significantly differ in terms of barriers at posttraining, F (2, 142) = 1.82, p = .17. Across conditions, the most common barriers were: (a) no regular clinical supervision available to support skills learned in training (M = 2.59, SD = 1.13), (b) no formal commitment from my organization to learning and implementing course content (M = 2.17, SD = 1.04), and (c) no relief time from work provided for learning and implementing course content (M = 2.08, SD = 1.01). Primary Outcome Analyses Table 2 presents descriptive data for the longitudinal outcome total scores (chain analysis and validation strategies content areas combined) at each time point, by condition, and the associated statistics from the HLM analyses. Between-condition Cohen’s d effects sizes are shown in Table 3. Table 2. Means, Standard Deviations, and HLM Results for Longitudinal Outcomes: Total Scores Baseline 90 Day Follow-up OLT ILT TM OLT ILT TM OLT ILT TM Outcome M (SD) n M (SD) n M (SD) n M (SD) n M (SD) n M (SD) n M (SD) n M (SD) n M (SD) n Self-efficacy 2.24 (0.67) 55a 2.54 (0.62) 55ab 2.13 (0.60) 62b 2.95 (0.54) 35c 3.43 (0.65) 52cd 2.87 (0.53) 58d 2.82 (0.60) 33e 3.38 (0.66) 46ef 2.77 (0.80) 50f Motivation 4.24 (0.50) 55 4.21 (0.48) 55 4.17 (0.50) 62 4.01 (0.48) 35c 4.23 (0.45) 52cd 3.98 (0.51) 58d 3.93 (0.58) 33 4.12 (0.57) 46e 3.74 (0.57) 50e Knowledge 0.38 (0.08) 55 0.37 (0.08) 55 0.38 (0.09) 62 0.64 (0.13) 35cd 0.58 (0.12) 51c 0.56 (0.13) 58d 0.60 (0.12) 31ef 0.53 (0.12) 46e 0.54 (0.14) 50f Proficiency 0.83 (0.62) 55 0.85 (0.55) 55 0.83 (0.59) 60 1.56 (0.63) 31 1.47 (0.59) 50 1.47 (0.75) 55 1.55 (0.76) 28 1.38 (0.59) 43 1.51 (0.80) 51 30 Day Follow-up 60 Day Follow-up 90 Day Follow-up Clinical use 2.40 (1.18) 29 2.16 (1.11) 50 2.73 (1.43) 55 2.47 (1.10) 30 3.02 (1.31) 49 2.46 (1.26) 55 2.52 (1.23) 31 2.72 (1.31) 46 2.48 (1.39) 51 Note. OLT = online training. ILT = instructor led training. TM = treatment manual. ab In a row, when means share the same subscript this means that they are significantly different in HLM analyses (p < .05) Table options Table 3. Cohen’s d Between-Condition Effect Sizes OLT vs. TM ILT vs. TM OLT vs. ILT Outcome Pre Post 90 Day FU Pre Post 90 Day FU Pre Post 90 Day FU Self-Efficacy Total 0.17 0.15 0.07 0.67⁎ 0.96⁎ 0.84⁎ 0.46⁎ 0.80⁎ 0.89⁎ Chain analysis 0.07 0.17 0.02 0.47⁎ 0.73⁎ 0.77⁎ 0.38⁎ 0.89⁎ 0.87⁎ Validation 0.17 0.36 0.24 0.52⁎ 0.92⁎ 0.70⁎ 0.36 0.56⁎ 0.49⁎ Motivation Total 0.14 0.06 0.34 0.08 0.52⁎ 0.67⁎ 0.06 0.45⁎ 0.34 Chain analysis 0.07 0.14 0.34 0.00 0.59⁎ 0.82⁎ 0.07 0.65⁎ 0.44 Validation 0.05 0.25 0.47 0.12 0.52 0.52 0.08 0.31 0.08 Knowledge Total 0.00 0.62⁎ 0.46⁎ 0.12 0.16 0.08 0.13 0.49⁎ 0.59⁎ Chain analysis 0.00 0.49 0.30 0.00 0.00 0.09 0.00 0.53 0.40 Validation 0.00 0.58⁎ 0.45⁎ 0.00 0.20 0.09 0.00 0.40⁎ 0.62⁎ Proficiency Total 0.00 0.09 0.05 0.04 0.00 0.18 0.03 0.10 0.26 Chain analysis 0.28 0.18 0.06 0.16 0.11 0.03 0.11 0.08 0.10 Validation 0.23 0.06 0.05 0.09 0.10 0.31 0.15 0.19 0.35 Note. Raw data were used to calculate effect sizes. Interpretation of Cohen’s d: .20 (small), .50 (medium), and .80 (large) ( Cohen, 1988). Pre = baseline. FU = follow-up. OLT = online training. ILT = instructor led training. TM = treatment manual. ⁎ Between-condition differences are significant at p < .05 in the HLM analyses. Table options Satisfaction Cross-sectional analyses at posttraining showed a significant between-condition difference on satisfaction with the training style and presentation methods, F (2, 142) = 9.99, p < .0001. ILT participants rated the training as more satisfactory than those in OLT, t(142) = 2.82, p < .01, and TM, t(142) = 4.38, p < .0001. Overall, participants reported the training styles and methods to be on average between “good” and “very” satisfactory: ILT (M = 4.01, SD = 0.64), OLT (M = 3.57, SD = 0.76), and TM (M = 3.41, SD = 0.74). Self-Efficacy A significant Time effect indicated that all conditions reported an increase in self-efficacy over time, F(2, 279) = 140.63, p < .001. A significant Condition effect, F(2, 164) = 13.95, p < .001, indicated differences between conditions across time points. A nonsignificant Time × Condition interaction effect, F(4, 277) = 0.76, p = .55, indicated that self-efficacy did not change differentially by condition across time. Post-hoc comparisons indicated that ILT participants rated their self-efficacy as significantly higher at baseline, posttraining, and final follow-up compared to TM participants, t(283) = 3.41, p < .001; t(296) = 4.53, p < .001; t(326) = 4.61, p < .001; respectively. Moreover, ILT participants reported greater self-efficacy compared to OLT participants at baseline, posttraining, and final follow-up, t(283) = 2.42, p < .05; t(339) = 3.18, p < .01; t(356) = 3.31, p < .001; respectively. No differences were found between OLT and TM conditions at any time point (p’s > .30). Given the significant baseline differences, the Condition effect is best interpreted as indicating that ILT participants began the study with higher self-efficacy and their scores remained higher than both other conditions over time. Motivation A significant effect for Time indicated the sample as a whole experienced a decrease in motivation over time, F(2, 142) = 16.51, p < .0001. Moreover, a significant Time × Condition interaction indicated differential change over time by condition, F(4, 141) = 2.80, p < .05. Post-hoc comparisons revealed participants in the ILT condition reported higher motivation at the posttraining and final follow-up assessments compared to the TM, t(143) = 2.34, p < .05; t(139) = 3.16; p < .005; respectively. Additionally, ILT participants reported higher motivation compared to OLT participants at posttraining, t(148) = 2.09, p < .05, but not at final follow-up, t (141) = 1.51, p = .13. Motivation started and remained high for all conditions throughout the study despite the significant decrease over time (means ranged from 3.74–4.24 on a 1–5 Likert scale). Knowledge At baseline, participants answered between 37%–38% of questions correctly. A significant effect for Time indicated the sample as a whole generally improved on knowledge across time, F(2, 287) = 344.49, p < .0001. Additionally, a significant Time × Condition interaction effect, F(4, 286) = 4.14, p < .01, indicated differential knowledge gains by condition over time. Post-hoc comparisons indicated that OLT participants demonstrated greater knowledge at the posttraining and final follow-up assessments compared to TM (t[342] = 3.97, p < .0001; t[363] = 2.45; p < .05; respectively) and ILT (t [337] = 2.57, p < .05; t [357] = 2.42; p < .05; respectively). Knowledge test scores ranged from 53% to 64% correct across conditions at the posttraining and final follow-up assessments. Clinical Proficiency A significant effect for Time indicated general improvement for the sample in ability to perform the treatment strategies in role plays, F(2, 156) = 57.24, p < .0001. The Time × Condition interaction was not significant, F(4, 155) = 0.47, p = .76, indicating there was no differential change between the conditions over time. Means at the posttraining and final follow-up assessments indicated moderate proficiency following training. Clinical Use A significant effect for Time indicated increased clinical use of the treatment strategies following training, F(2, 130) = 4.26, p < .05. The Condition effect was not significant, F (2, 127) = 0.73, p = .48, indicating there was no overall difference between conditions over the follow-up period. Descriptive data indicate that participants used chain analysis and/or validation strategies on average between 1 to 9 times during each 30-day follow-up period. Exploratory Outcome Analyses Exploratory analyses were conducted to compare training conditions separately by content area taught (chain analysis vs. validation strategies). For the longitudinal outcomes, between-condition Cohen’s d effects sizes are shown in Table 3 and results of HLM analyses are presented in Table 4 by content domain. These analyses indicated that all three conditions resulted in improved self-efficacy across time points for each content area. ILT was superior to TM and OLT at every time point regardless of the content area being taught, with the exception of baseline for validation strategies (ILT was superior to TM, but not OLT). Similarly, ILT was superior in increasing motivation to use chain analysis compared to TM (at both posttraining and 90-day follow-up) and OLT (at posttraining only). The Time × Condition interaction effect was not significant for motivation to use validation strategies. OLT was particularly effective at increasing knowledge of validation strategies, with OLT outperforming ILT and TM at both outcome points. The Time × Condition interaction effect was not significant for knowledge of chain analysis. There were no between-condition differences for either chain analysis or validation strategies in clinical proficiency and no clear pattern of results emerged for self-reported use of the strategies in clinical practice. Table 4. Means, Standard Deviations, and Exploratory HLM Results for Longitudinal Outcomes: Content Domain Specific Baseline Post-Training 90 Day Follow-up OLT ILT TM OLT ILT TM OLT ILT TM Fdf M (SD) M (SD) M (SD) M (SD) M (SD) M (SD) M (SD) M (SD) M (SD) Condition Time Condition x Time Self-Efficacy Chain analysis 1.92 (0.75)a 2.24 (0.76)ab 1.90 (0.70)b 2.79 (0.61)c 3.43 (0.80)cd 2.90 (0.67)d 2.58 (0.84)e 3.30 (0.84)ef 2.60 (0.98)f 11.922,167*** 142.692,283*** 1.654,282 Validation 2.28 (0.65) 2.52 (0.69)a 2.17 (0.67)a 3.16 (0.76)c 3.55 (0.70)cd 2.91 (0.71)d 3.05 (0.88)e 3.43 (0.72)ef 2.82 (1.01)f 10.732,170*** 113.422,288*** 1.174,287 Motivation Chain analysis 3.95 (0.66) 3.90 (0.71) 3.90 (0.69) 3.84 (0.61)c 4.14 (0.65)cd 3.75 (0.68)d 3.65 (0.77) 3.97 (0.72)e 3.41 (0.66)e 4.012,155* 12.992,144*** 3.294,143* Validation 4.20 (0.63) 4.25 (0.66) 4.17 (0.63) 4.09 (0.53) 4.26 (0.58) 3.93 (0.70) 4.02 (0.72) 4.08 (0.82) 3.66 (0.81) 3.342,155* 7.812,169*** 1.354,169 Knowledge Chain analysis 0.47 (0.10) 0.47 (0.09) 0.47 (0.10) 0.67 (0.09) 0.62 (0.10) 0.62 (0.11) 0.63 (0.09) 0.59 (0.11) 0.60 (0.11) 1.182,166 264.652,278*** 1.634,278 Validation 0.24 (0.10) 0.24 (0.10) 0.24 (0.10) 0.59 (0.21)cd 0.51 (0.20)c 0.47 (0.21)d 0.55 (0.19)ef 0.44 (0.19)e 0.46 (0.22)f 5.302,169** 182.682,297*** 3.104,296* Proficiency Chain analysis 0.65 (0.62) 0.58 (0.63) 0.48 (0.60) 1.52 (0.77) 1.46 (0.73) 1.38 (0.80) 1.50 (0.79) 1.42 (0.82) 1.45 (0.97) 0.682,150 92.332,141*** 0.374,139 Validation 1.00 (0.79) 1.11 (0.71) 1.18 (0.79) 1.61 (0.72) 1.48 (0.68) 1.56 (0.90) 1.61 (0.88) 1.35 (0.65) 1.57 (0.78) 0.792,167 18.782,291*** 0.884,289 30 Day Follow-up 60 Day Follow-up 90 Day Follow-up Clinical use Chain analysis 2.04 (1.09) 1.80 (1.07) 2.40 (1.42) 1.93 (1.08) 2.49 (1.16) 2.00 (1.16) 2.21 (1.18) 2.35 (1.16) 2.08 (1.13) 0.412,133 1.302,250 8.094,250*** Validation 2.78 (1.55) 2.52 (1.27) 3.04 (1.78) 2.93 (1.41) 3.54 (1.86) 2.94 (1.80) 2.86 (1.58) 3.09 (1.75) 2.92 (1.98) 0.572,137 4.922,148** 4.524,150** Note. For outcomes where baseline scores were covaried (i.e., self-efficacy), between condition effects are represented by the Condition F value. For all other outcomes, between condition effects are represented by the Time x Condition F value. The number of participants who completed the posttraining assessment varied somewhat by measure: OLT (n = 31–35), ILT (n = 50–52), TM (n = 55–58). The number of participants who completed the 90 day follow-up assessment varied somewhat by measure: OLT (n = 28–33), ILT (n = 43–46), TM (n = 50–51). *p < .05, **p < .01, ***p < .001. ab In a row, when means share the same subscript this means that they are significantly different (p < .05).