This study expands and improves the methodologies available to evaluate diagnostic medical technologies. With these improved methods, subsequent studies of numerous diagnostic medical technologies should prove more accurate and less biased results than current methods allow. As a major part of this project, we will develop, document, and disseminate software that takes advantage of the methodological improvements made during this study after extensive """"""""tuning"""""""" to several large data sets made available to the investigators. These studies focus on the estimation of receiver operating characteristic (ROC) curves, the universally preferred approach to evaluating the efficacy of diagnostic tests. The study contains three major projects. Project I addresses problems associated with the estimation of ROC curves using ordinal data, including further development of confidence bands around ROC curves, methods to eliminate small-sample bias inherent in maximum likelihood estimation methods and improved estimates of standard errors of ROC curve parameters. This project also extends the ROC problem from a one-disease plus normal case to the M-disease plus normal case (e.g., disease staging), investigating the appropriate methods for measuring accuracy of such tests and developing ways to describe samples needed for estimation of these ROC surfaces. Project II addresses a common problem in ROC curve estimation, namely, the proper approach when the """"""""gold standard"""""""" contains error of unknown magnitude. We expand and test the effectiveness of two approaches to dealing with this problem that we developed in a previous study of diagnostic technologies funded by AHCPR, and propose a third new maximum likelihood estimator that takes account directly of the imperfect nature of the gold standard. These all offer different approaches to resolving the problem of an inaccurate gold standard. We will test these approaches using Monte Carlo studies as well as deriving analytic statements about the improvement in ROC accuracy where possible. Project III addresses two alternative ways of constructing ROC curves; for one, the more familiar logit regression approach, we deal 4th ways to improve the estimates of standard errors around ROC parameters; previous approaches to this provide biased standard errors. We also will study the construction of ROC curves using classification trees a new approach that has considerable potential appeal in estimation of ROC curves.
| Peng, F; Hall, W J (1996) Bayesian analysis of ROC curves using Markov-chain Monte Carlo methods. Med Decis Making 16:404-11 |
