Anticipated Impacts on Veterans Healthcare Findings from this study will be valuable in identifying quality gaps in AMI patient outcomes across VA hospitals. They will also provide valuable guidance for other VA programs, such as the External Peer Review Program (EPRP), that monitor and evaluate hospital outcomes. Implementation of this study will be fruitfully linked with an ongoing HSRD-funded study (IIR 08- 351, PI: Hanchate) using enriched electronic patient data for improving risk adjustment models in the VA. We view this pilot study as the basis for a full-scale project to explore a range of inpatient cohorts, in partnership with an ongoing VA quality monitoring program to facilitate adoption of appropriate statistical methods and measures. By examining novel statistical methods to 'analyze veteran record data fields for performance and quality assessments' this study addresses an important VA priority research area (Priority J). Project Background Crucial to the ongoing drive toward improving VA healthcare quality is the ability to assess performance. While VA has made great strides in automating and systematizing patient data, 'there is a paucity of research into the ability of different statistical methods to correctly identify excellent, average, or poor performers'. For instance, while O/E - the ratio between observed and expected rate of an outcome - has been the traditional reporting measure of hospital performance, CMS's HospitalCompare program of comparing hospitals nationally uses an alternative measure, P/E, where the numerator (P) is the hospital-specific risk-adjusted estimate from a hierarchical regression model. Little research has gone into the relative assessment of these two methods/measures, or more generally, in identifying the most accurate and reliable statistical methods and measures to identify performance. Project Objectives Hospital performance on three inpatient outcomes (inpatient mortality, 30-day rehospitalization and length of stay) will be separately evaluated using comprehensive patient clinical information from inpatient, outpatient, laboratory test results, vital signs and vital status data files (FY 2004- 2009). The specific objectives are to: 1. Evaluate the accuracy and reliability of alternative statistical methods and measures of hospital performance in simulated data, and 2. For selected methods and measures with higher accuracy and reliability, estimate performance of VA hospitals using AMI inpatient outcomes data. Project Methods Preliminary examination of VA hospital performance in terms of inpatient mortality following admissions for acute myocardial infarction (AMI) indicates considerably different profiling of most hospitals depending on the profiling measure. We also found sizable estimation uncertainty from both methods, making reliable identification of outliers difficult. As such there is a clear need to evaluate alternative methods (logistic, Poisson, linear probability, Bayesian models) and measures, in terms of both accuracy and reliability. No previous study has made such a comprehensive and systematic evaluation. The primary approach to this evaluation will be the development of a series of Monte Carlo simulated data sets tailored to capture important characteristics of patient risk factors and outcomes in the VA - including variation in hospital outcome and risk factor rates and hospital volumes. As true hospital performance is predetermined by design in the development of simulated data, it serves as the reference ('gold') standard in evaluating accuracy and reliability of the alternative measures and methods.
While VA has made great strides in automating and integrating patient data, there is a paucity of research into the ability of different statistical methods to correctly identify excellent, average, or poor performers. Using Monte Carlo simulation methods, this study will evaluate a range of alternative methods and measures of hospital performance using inpatient outcomes (30-day mortality, 30-day readmission and length of stay) for veterans admitted for acute myocardial infarction (AMI). Using administrative patient data (FY2004-2009) our aim will be to identify the most accurate and reliable methods and measures for identifying performance outliers. Findings from this study will be valuable in identifying quality gaps in AMI patient outcomes across VA hospitals. They will also provide valuable guidance for other VA programs, such as the External Peer Review Program (EPRP), that monitor and evaluate hospital outcomes
| Hanchate, Amresh D; Stolzmann, Kelly L; Rosen, Amy K et al. (2017) Does adding clinical data to administrative data improve agreement among hospital quality measures? Healthc (Amst) 5:112-118 |
