Background and Anticipated Impacts on VA Patient Care: Our recently completed HSR&D-funded research (IIR 07-092-1) revealed three serious problems with the validity of Veterans Health Administration (VHA) and Health Plan Employer Data and Information Set (HEDIS) measures of addiction treatment quality: First, the presumed associations between established process quality measures and outcomes are weak or non-existent. Second, the validity of underlying strategies used at VHA and HEDIS for identifying health care encounters in the medical record is generally poor and varies substantially by setting and facility. Third, currently utilized addiction treatment quality measures focus exclusively on process and do not adequately capture other important domains of quality (e.g., structure, access, patient experiences, and outcomes). All VHA stakeholders benefit from the existence and proper use of valid healthcare quality measures. In light of the now recognized need to improve and expand addiction-related quality measurement, the proposed study has the following objectives.
Aim 1. Evaluate the Associations between Currently Proposed, High-Profile Measures of Addiction Treatment Quality and Outcomes (predictive validity): Predictive validity refers to the strength of association between access or process quality measures and subsequent patient outcomes. We identified three sources of 41 newly developed, but unvalidated, addiction treatment quality measures: 18 VA Unform Mental Health Services Handbook Metrics, 18 from the recently completed congressionally-mandated RAND/Altarum evaluation of VA mental health services;and 5 newly developed metrics from the Washington Circle policy group.
Our aim i s to evaluate the associations between these measures and outcomes, in four samples of patients for whom we have pre-existing outcome data.
Aim 2. Examine the Associations Among Contemporaneously Measured Quality Indictors and Investigate the Validity of Underlying Care Identification Strategies (Concurrent Validity). According to Donabedian, quality indicators should be associated to other contemporaneously measured indicators of the underlying construct. If the association between theoretically linked quality indicators is strong, this is evidence that the method of assessment is strong and the subsequent inferences are valid. Conversely if the expected relationships between indicators are weak, this suggests problems with the quality or measurement model. Therefore, we will assess the associations between contemporaneously measured, theoretically linked quality measures. Another aspect of concurrent validity is the extent to which quality measure specifications accurately identify the targeted processes and patients in available data (termed specification strategy). Our previous HSR&D-funded research found several problems in the specification strategies used in both the VHA Continuity of Care measure (based on VHA clinic stop codes), and the 2006 version of HEDIS Initiation and Engagement measures (based on diagnosis/CPT code combinations). We propose to conduct a similar validation study of the substantially different 2010 HEDIS specification strategy, as well as specification strategies used in the new OMHS metrics. Secondary Aim1. Evaluate the Impact of Diagnostic and Setting Factors that May Moderate Predictive or Concurrent Validity. Given the finding by our team and others that relationships between patterns of care and outcomes vary by diagnostic and setting factors, we will evaluate these potential moderators of the predictive and concurrent validity examined in Aims 1 and 2. Secondary Aim 2. Evaluate Methods to Combine Validated Quality Metrics into Information that is Clinically and Operationally Meaningful. We propose to explore various means of combining the quality metrics into composite measures, and then evaluate the association of these composites with outcomes.
Current measures of addiction treatment quality have been shown to have multiple validity problems. Tightly alligned with the Offices of Mental Health Services and Mental Health Operations priorities, this project aims to validate 41 newly developed measures of addiction treatment quality. For clinicians and clinical managers, validated quality measures help define and motivate guideline-congruent care, can identify gaps in the continuum of care, and can help identify high and low performing facilities so that quality improvement efforts can be targeted. Validated measures of addiction treatment quality also provide veterans with a means to compare the quality of care provided at different health care facilities. For VA policy makers, validated quality measures provide a means to document systemic improvement or deterioration and may act as important program evaluation outcomes.
| Schmidt, Eric M; Gupta, Shalini; Bowe, Thomas et al. (2017) Predictive validity of a quality measure for intensive substance use disorder treatment. Subst Abus 38:317-323 |
| Harris, Alex H S; Weisner, Constance M; Chalk, Mady et al. (2016) Specifying and Pilot Testing Quality Measures for the American Society of Addiction Medicine's Standards of Care. J Addict Med 10:148-55 |
| Harris, Alex H S (2016) The primitive state of quality measures in addiction treatment and their application. Addiction 111:195-6 |
