The purpose of this career development award is to develop a research and training program that will enable Dr. Glance to become an independent physician investigator in the field of health outcomes research. The focus of this proposal will be to examine the feasibility of using risk-adjustment models for benchmarking Intensive Care Unit (ICU) performance in a large multi-institutional database. Crude mortality rates cannot be used to compare ICUs because they do not account for differences in the severity of disease between ICU patient populations. However, the Standardized Mortality Ratio (SMR), defined as the ratio of the observed mortality rate (OMR) to the expected mortality rate (EMR), does account for differences in patient case mix between ICUs, and can therefore be used to compare the performance of individual ICUs. The EMR is obtained by calculating the predicted mortality of the patients in a specific population using an ICU scoring system. Using severity-adjusted outcome measures like the SMR, low- and high-performance ICUs can be identified. A potential problem with this approach is the finding that general severity scoring systems disagree on how to rank individual hospitals using severity-adjusted outcome measures. It is likely that ICU scoring systems will behave in the same manner;
Specific Aim #1 will examine this hypothesis for APACHE 11 (Acute Physiology and Chronic Health Evaluation), SAPS 11 (Simplified Acute Physiology Score) and MPM II (Mortality Probability Model). A finding that different scoring systems disagree on the identity of quality outliers raises doubt regarding the face validity of benchmarking efforts using severity-adjusted outcome measures. Validation studies have demonstrated that ICU scoring systems perform poorly when evaluated in most data sets (other than the one used to develop the original model). Therefore, it not surprising that severity-adjusted outcome measures do not agree on hospital quality when the scoring systems on which they are based are not accurate. However, predictive models can be customized to a data set so that they will more accurately predict outcomes within that data set.
Specific Aim #2 will examine the proposition that ICU scoring systems are more likely to agree on the identity of quality outliers after the scoring systems have been customized to the ICU database made up of the ICUs whose performance is being benchmarked.
