Poor implementation explains significant reductions in effect sizes when Evidence-Based Programs (EBPs) are translated to real-world settings (Henggeler, 2004). Multiple dimensions of implementation influence outcomes (Durlak & DuPre, 2008), but only four occur within the delivery of program sessions and, as a result, are potential modifiable sources of disconnect between the program as designed and as implemented: 1) fidelity (amount of curriculum providers deliver), 2) positive adaptations (the quality of unscriptd additions providers make during delivery), 3) quality of delivery (skill with which providers delivr material and interact with participants, and 4) participant responsiveness (level of program participation, such as participant attendance and program skill use). Identifying efficient methods to monitor these 4 dimensions (Schoenwald, et al., 2011) as well as understanding how they interrelate to influence outcomes (Berkel, Mauricio, et al., 2011) and change across sessions has important implications for maintaining EBP effects in real-world settings. The proposed research uses data from the NIDA-funded effectiveness trial of the New Beginnings Program (NBP; R01DA026874) and independent observer data that we will collect. NBP is a parenting intervention that improves youth substance use and mental health outcomes in response to parental divorce (Wolchik, et al., 2002) through effects on intervention-targeted parenting mediators (McClain, et al., 2010). Community providers will deliver the NBP to approximately 550 parents across 75 intervention groups through a partnership with 5 county-level courts in Arizona. The effectiveness trial will only collect independent observer data on 10% of sessions and only examine simple, direct effects of implementation on outcomes. This study builds on the NBP trial by collecting independent observer data for 100% of sessions, providing the required data to address 3 research aims that will inform critical questions in implementation measurement and theory.
In Aim 1, we compare the reliability and predictive validity of fidelity, positive adaptation, and quality across independent observer ratings, provide self-report, and supervisor ratings. We also examine sampling strategies to identify the optimal quantity of data (e.g., 10%, 20%) that must be sampled to reliably assess fidelity, positive adaptations, and quality and if this is contingent on program activity type (e.g., didactic versus skills practice) or provider characteristics (e.g., experience, attitudes about EBPs).
In Aim 2, we test theoretical frameworks explaining implementation effects on outcomes and examine the role of parent gender and ethnicity within these frameworks.
Aim 2 builds on extant studies by testing: a) moderational and meditational hypotheses about the effects of provider behaviors (i.e., fidelity, adaptation, quality) on responsiveness and outcomes, b) the temporal precedence of provider behavior effects on responsiveness and outcomes, and c) the moderating effects of parent gender and ethnicity on model pathways.
In Aim 3, we examine changes in fidelity, adaptation, and quality over time and the influence of provider characteristics on these changes.
Evidence Based Programs (EPBs) can prevent substance use, which is associated with economic costs that exceed $484 billion and more than 2 million American deaths each year. Implementation failure causes significant reductions in effect sizes and public health impact when interventions targeting substance abuse are translated to real-world settings. The proposed research has 3 aims with important public health implications for disseminating EBPs: 1) efficient methods to assess implementation, 2) the mechanisms by which multiple dimensions of implementation interrelate to influence program outcomes and 3) how implementation dimensions change across sessions and the effects of provider characteristics on these changes.
| Berkel, Cady; Mauricio, Anne M; Sandler, Irwin N et al. (2018) The Cascading Effects of Multiple Dimensions of Implementation on Program Outcomes: a Test of a Theoretical Model. Prev Sci 19:782-794 |
| Mauricio, Anne M; Mazza, Gina L; Berkel, Cady et al. (2018) Attendance Trajectory Classes Among Divorced and Separated Mothers and Fathers in the New Beginnings Program. Prev Sci 19:620-629 |
| Berkel, Cady; Sandler, Irwin N; Wolchik, Sharlene A et al. (2018) ""Home Practice Is the Program"": Parents' Practice of Program Skills as Predictors of Outcomes in the New Beginnings Program Effectiveness Trial. Prev Sci 19:663-673 |
