In order to develop a learning health care system (LHCS), VHA leadership must understand where quality improvement is needed via valid and actionable performance measurement and reporting. Performance measurement that serves as an effective tool for systemwide-learning is based on empirical evidence supporting the reliability and validity of measures at each level of decision making, a data-warehouse that provides timely access to relevant data at multiple levels and across multiple different time spans, an analytics engine for processing data and generating actionable information, and an effective reporting system for delivering timely information to the appropriate stakeholders. In addition, a clear focus on outcomes avoids the problem stemming from the proliferation of process measures that reduce the ratio of ?signal? (important outcomes) to ?noise? (process measures of marginal value). The VHA has developed a variety of methods and measures to capture clinical information and to assess health care quality. Introduced in 2012, the Strategic Analytics for Improvement and Learning Value (SAIL) report provides facility performance information on 28 performance metrics. The SAIL report focuses on facility-level variability across diverse performance metrics. However, there is growing evidence that variation in patient outcomes is greatest at lower levels of the health system. In preliminary work for this application we found similar patterns in employee data. We found that workgroups at the nursing unit level explain a significant proportion of variation in employee satisfaction. At the same time, variability in satisfaction at the facility level was nearly zero. This means that important within-hospital unit-level differences in satisfaction are obscured by a focus upon the facility level as a unit of analysis and reporting. Therefore, sites cannot be distinguished in the basis of average employee satisfaction. Based upon the literature in health care and other fields such as education, we anticipate that this same phenomenon will hold for the outcomes we will analyze. In contrast, the SAIL report, with its reliance on facility-level outcomes and measures, assumes that facility- level variability is reliable while ignoring the contributions of unit-level variance. These assumptions reflect the concept of ecological fallacy and demonstrate a need in the VHA for an analytical model that can provide valid performance information by assessing variation at multiple levels of the health system. Our goal for this project is to advance the science of multi-level health care performance measurement and feedback to support a LHCS. We will build an analytical model that provides a valid and reliable assessment of inpatient outcomes and their structural predictors at multiple levels of the health system, and we will present this data in feedback reports targeted to those front-line clinicians and administrators who can use the results to improve the quality of care. To achieve this goal, we will 1) build a multi-level structural equations model (ML-SEM) using inpatient outcomes (mortality, readmissions, adverse events) and their predictors (e.g. patient disease burden, staffing levels) to simultaneously evaluate variation at the unit level and facility level; and 2) develop templates for displaying facility performance data that are tailored to stakeholder needs and facilitate quality improvement. Constructing a model to assess variation at multiple levels (Aim 1) will begin by using a mixed-effects model to examine variation in outcomes and predictors. Next, we will use a predictive model to identify significant predictors of outcomes. Finally, developing reports using our analytical model results (Aim 2) will use a mixed-methods approach encompassing stakeholder needs assessment and iterative design and usability pilot testing. Our goal is to advance the science of measurement beyond crude measures of overall facility and VISN performance, toward more actionable feedback about sources of variability in performance. This work will meet the needs of a LHCS by leveraging the vast VHA data infrastructure to generate valid and actionable knowledge and effectively conveying it to end users for improving the quality of care for Veterans.
In order for the VHA to become a learning health care system, it needs to leverage its data resources to provide valid and actionable information to leaders at multiple levels of the VHA system, not just at the facility level. Our goal is to advance the science of performance measurement beyond crude measures of overall facility and VISN performance, toward more actionable feedback that integrates work unit-level sources of variability in performance. We will use novel computational methods and evidence-based design of performance feedback to tailor information to the needs and sphere of influence of the recipient. Our analytical model and reports will allow decision makers at all levels within the VHA to be informed about the quality of care they provide and underlying factors, such as resource availability and service burden, which affect quality and patient outcomes. Our goal is to enable continuous and effective quality improvement throughout the VHA as part of a learning health care system and to ensure the best possible care for Veterans.
