The goal of proposed project is to strengthen the public health impact of psychosocial interventions by identifying fidelity measurement methods that can be used for research and practice. To date, fidelity has posed a thorny measurement quandary because few reliable, valid, and efficient fidelity measurement methods exist. The gold standard for measuring fidelity to psychosocial interventions, direct observation of therapist behavior, requires extensive resources. When fidelity is measured in the community, the most commonly used and least resource intensive method is self-report. Unfortunately, concordance between observation and self-report is low. There is a critical need to identify and evaluate methods of fidelity measurement that are accurate (i.e., measure what they intend) and cost-effective. Our objective in this measurement proposal is to compare the accuracy and cost-effectiveness of self-report (fidelity measurement- as-usual) and two innovative methods (chart stimulated recall and behavioral rehearsal) in assessing fidelity to cognitive-behavioral therapy, an established evidence-based practice. We will randomize 135 therapists, trained in cognitive-behavioral therapy through an initiative to implement evidence-based practices in the City of Philadelphia, to 3 conditions: self-report (N = 45), chart stimulated recall (N = 45), and behavioral rehearsal (N = 45). All conditions will include direct observation using the Therapy Process Observational Coding System as the gold-standard comparison.
In Aim 1, we will identify the most accurate fidelity measurement method.
In Aim 2, we will estimate the economic costs and cost-effectiveness of the proposed fidelity measurement methods.
In Aim 3, we will compare stakeholders' willingness to use each method, as well as identify their perceived barriers and facilitators to use of each method. The proposed work is consistent with the NIMH strategic plan, specifically Objective 4, to strengthen the public health impact of NIMH-supported research. This study will have a significant positive impact by producing fidelity measurement methods that can then be used by implementation scientists for research and community mental health clinics to monitor therapist fidelity, an indicator of therapy quality.
The work proposed has the potential to impact public health by identifying accurate and cost-effective fidelity measurement methods that can be used for both research and practice. Fidelity is the most measured implementation outcome in implementation science trials, and yet we do not have accurate and cost-effective ways to measure fidelity. Further, with the passing of the Affordable Care Act, agency monitoring of therapy quality will be mandated. Identifying accurate and cost-effective ways for community mental health agencies to measure fidelity is at a critical juncture.
| Lane-Fall, Meghan B; Cobb, Benjamin T; Cené, Crystal Wiley et al. (2018) Implementation Science in Perioperative Care. Anesthesiol Clin 36:1-15 |
| Beidas, Rinad S; Maclean, Johanna Catherine; Fishman, Jessica et al. (2016) A randomized trial to identify accurate and cost-effective fidelity measurement methods for cognitive-behavioral therapy: project FACTS study protocol. BMC Psychiatry 16:323 |
