The proposed pair of studies (R21/R33) addresses the need to effectively and efficiently assess and support the fidelity of empirically supported behavioral treatments (ESBTs) in community based clinical practice. The studies aim to develop and evaluate a relatively low-burden observational instrument to index the adherence and competence of clinical supervisors; and to evaluate the effects of web-based feedback from the instrument on supervisor performance, therapist adherence, and post-treatment outcomes of treatment, using Multisystemic Therapy (MST) as an example. In the R21, clinical supervisors in MST programs, MST trainers, and the investigators will together define supervisor adherence and competence, specify those components of each construct specific to MST and those likely pertinent to clinical supervision of other (ESBTs), and develop items indexing these components. Then, for 10 weeks, 30 community-based clinical supervisors in MST programs will digitally record and upload to a secure website weekly group supervision sessions. The sessions will be coded and the resulting data evaluated for reliability and validity using Item Response Theory approaches. Multilevel modeling will be used to evaluate associations among supervisor adherence, supervisor competence, and therapist adherence. In the R33, the individuals who helped develop the supervisor adherence and competence instrument will collaborate with the investigators and a web designer to develop a web-based format for providing feedback to supervisors on the instrument developed in the R21. Then, 60 community-based clinical supervisors in MST programs will digitally record and upload weekly supervision sessions for seven months, and half will be randomly assigned to receive monthly web-based feedback about their adherence and competence ratings. Feedback effects will be evaluated on supervisor performance, therapist adherence, and immediate post-treatment youth outcomes. As a step toward maximizing the efficiency of instrument use, effects will be evaluated of whole session and session segment coding. Estimates will be developed of the resources required to utilize the instrument and web-based feedback intervention in practice. This pair of projects will identify aspects of clinical supervision linked with treatment adherence and outcomes that can be improved in real time and an efficient mechanism to support that improvement. Results will inform research to identify aspects of ESBT training and implementation support clinical supervisors can ably execute and strategies and technologies that facilitate their doing so. Such research is aimed at efficiently enhancing the capacity of the community based mental health workforce to sustain ESBT implementation, thereby potentially increasing the market penetration of ESBTs and thus of consumer access to them.
Increasing the use and fidelity of empirically supported behavioral treatments (ESBTs) in community practice settings has the potential to have a large, positive, public health impact. The proposed studies aim to identify the aspects of clinical supervision associated with practitioner adherence and client outcomes that can be improved in real time, and an efficient mechanism to support that improvement. In collaboration with the organizations that provide training and support to community-based providers of Multisystemic Therapy (MST) and community-based clinical supervisors, an observational measure of supervisor adherence and competence will be developed; and, a randomized trial will test the effects on supervision, therapist adherence, and youth post-treatment outcomes of providing supervisors web-based feedback about their performance on this measure.
