The importance of moving Evidence Based Practice (EBP) and specific mental health Evidence Based Interventions (EBIs) from research to practice is widely recognized. The field of implementation science has rapidly evolved to bridge this gap and is increasingly building evidence about influencing factors and implementation strategies for EBP and EBIs. Unfortunately, successful implementation does not guarantee ongoing sustainment. Without sustainment the public health benefits promised by any EBI will not be achieved and the accumulated costs from development, evaluation and implementation are wasted. Therefore, achieving sustainment is essential to strengthen the public health impact of research. Although a sustainment phase of implementation has been acknowledged for decades, research on sustainment and the mechanisms supporting it is limited by a lack of a pragmatic, reliable, and valid measure. There is particular need for a sustainment measure that assesses whether effective interventions continue to be used, have maintained capacity and persist in producing benefits. Current measures appear to be long, complex, context specific and unsuitable for completion by front-line service providers. As such, there is no measure that is consistently used across studies, thereby limiting replication and comparisons. We propose to review and extend the current work on sustainment definitions, frameworks and tools to develop a generalizable measure of EBI sustainment. The proposed work is innovative in developing a measure that will be generalizable across different health settings and organizational contexts and a wide-range of psychosocial EBIs. The measure will be administered to providers in mental health agencies, schools, substance use disorder treatment centers, and child welfare agencies where psychosocial EBIs are commonly delivered. In addition, the proposed work is innovative in its use of the Rasch Measurement Model, a member of the Item Response Theory family, to conduct the scale analyses together with the traditional Classical Test Theory statistical approaches. Specifically our aims are:
Aim 1 : Catalogue existing sustainment measurement constructs, generate an item pool, and assess items' face and content validity;
Aim 2 : Collect data and conduct reliability and construct validity analyses using classical test theory;
Aim 3 : Conduct Rasch analyses to assess item functioning; examine potential demographic, contextual and nested structure influences (e.g. agency, service system, experience, race and ethnicity) on item functioning; and determine cut-off points for scale scores;
Aim 4 : Examine convergent validity by assessing relationships between the sustainment measure and sustainment climate, sustainment leadership, and attitudes towards the specific EBI.
The proposed project aims to develop and test a measure of the important, but often forgotten, stage of the implementation process labeled sustainment. The team will catalogue current theoretical constructs of sustainment, develop relevant items, and test the psychometric properties of the resulting scale. The scale will facilitate measurement of the degree to which an evidence based intervention is integrated into practice with the organizational capacity to support its continued use.
