Receiver Operating Characteristic (ROC) analysis is recognized widely as the best way of measuring and specifying the accuracies of diagnostic procedures, because it is able to distinguish between actual differences in discrimination capacity, on one hand, and apparent differences that are due only to decision-threshold effects, on the other. Key methodological needs remain to be satisfied before ROC analysis can address all of the practically important situations that arise in diagnostic applications, however. This project employs signal detection theory and computer simulation to address several of those needs, by: (1) refining and continuing distribution of software developed previously by the applicants for fitting ROC curves and for testing the statistical significance of differences between ROC curve estimates; (2) developing and evaluating new algorithms for ROC curve-Fitting and statistical testing, based on their recently-developed """"""""proper"""""""" binormal model, that should provide more meaningful results in experimental situations that involve small samples of cases; (3) investigating the usefulness of a form of ROC methodology that is based on mixture distributions in order to rduce the need for diagnostic truth in ROC experiments; (4) investigating the effect of case-saple difficulty on the statistical power tests for differences between ROC curves, in order to determine the optimal difficulty of cases that shouldbe studied on rank diagnostic systems; and (5) developing methods for training artificial neural networks (ANNs) to maximize diagnostic accuracy in terms of ROC analysis and signal detection theory.

Agency
National Institute of Health (NIH)
Institute
National Institute of General Medical Sciences (NIGMS)
Type
Research Project (R01)
Project #
5R01GM057622-03
Application #
6181168
Study Section
Diagnostic Imaging Study Section (DMG)
Program Officer
Onken, James B
Project Start
1998-05-01
Project End
2003-09-30
Budget Start
2000-05-01
Budget End
2003-09-30
Support Year
3
Fiscal Year
2000
Total Cost
$218,176
Indirect Cost
Name
University of Chicago
Department
Radiation-Diagnostic/Oncology
Type
Schools of Medicine
DUNS #
225410919
City
Chicago
State
IL
Country
United States
Zip Code
60637
Jiang, Yulei; Metz, Charles E (2006) A quadratic model for combining quantitative diagnostic assessments from radiologist and computer in computer-aided diagnosis. Acad Radiol 13:140-51
Edwards, Darrin C; Lan, Li; Metz, Charles E et al. (2004) Estimating three-class ideal observer decision variables for computerized detection and classification of mammographic mass lesions. Med Phys 31:81-90
Edwards, Darrin C; Kupinski, Matthew A; Metz, Charles E et al. (2002) Maximum likelihood fitting of FROC curves under an initial-detection-and-candidate-analysis model. Med Phys 29:2861-70
Wagner, Robert F; Beiden, Sergey V; Campbell, Gregory et al. (2002) Assessment of medical imaging and computer-assist systems: lessons from recent experience. Acad Radiol 9:1264-77
Beiden, S V; Wagner, R F; Campbell, G et al. (2001) Components-of-variance models for random-effects ROC analysis: the case of unequal variance structures across modalities. Acad Radiol 8:605-15
Wagner, R F; Beiden, S V; Metz, C E (2001) Continuous versus categorical data for ROC analysis: some quantitative considerations. Acad Radiol 8:328-34