EXCEED THE SPACE PROVIDED. Assessment of diagnostic performance of technologies and practices constitute an increasingly important part of the decision making process when imaging systems are evaluated for possible use in the clinical environment as well as during the regulatory approval process. In medical imaging a substantial fraction of these evaluations include task specific ratings that are being analyzed using an ROC type response or derivative thereof. The overall objective of this project is to improve our understanding of the practical (methodological) and computational (analyses related) issues that we encounter during systems evaluations and comparisons when the observer, a computerized scheme (e.g. CAD), or a combination of both become an integral part of the diagnostic system. During the current period of funding we developed and tested a new approach to analyzing scoring data ascertained during the performance of ROC type studies and began to address issues related to the incorporation of relevance based utilities into the analyses. The underlying approach uses all paired observations between negative and positive cases (examinations) to construct summary indices of performance using a non-parametric methodology. Both unconditional and conditional tests have been developed and tested and were shown to have distinct experimental advantages for small sample size studies. Five primary efforts are proposed using this general approach. We propose to expand the method to a multi-reader, multi-modality, multi-disease settings as well as to incorporate a variety of utility functions into the analysis. Last, we will investigate how this and other methods can be used to identify and possibly treat outliers in ROC type studies. PERFORMANCE SITE ========================================Section End===========================================

Agency
National Institute of Health (NIH)
Institute
National Institute of Biomedical Imaging and Bioengineering (NIBIB)
Type
Research Project (R01)
Project #
5R01EB002106-11
Application #
6890900
Study Section
Diagnostic Imaging Study Section (DMG)
Program Officer
Haller, John W
Project Start
2002-05-01
Project End
2007-02-28
Budget Start
2005-05-01
Budget End
2007-02-28
Support Year
11
Fiscal Year
2005
Total Cost
$278,027
Indirect Cost
Name
University of Pittsburgh
Department
Radiation-Diagnostic/Oncology
Type
Schools of Medicine
DUNS #
004514360
City
Pittsburgh
State
PA
Country
United States
Zip Code
15213
Gur, David; Bandos, Andriy I; Rockette, Howard E et al. (2010) Is an ROC-type response truly always better than a binary response in observer performance studies? Acad Radiol 17:639-45
Gur, David; Rockette, Howard E (2009) Performance assessments of diagnostic systems under the FROC paradigm: experimental, analytical, and results interpretation issues. Acad Radiol 16:770-1
Gur, David; Bandos, Andriy I; Klym, Amy H et al. (2008) Agreement of the order of overall performance levels under different reading paradigms. Acad Radiol 15:1567-73
Gur, David; Bandos, Andriy I; King, Jill L et al. (2008) Binary and multi-category ratings in a laboratory observer performance study: a comparison. Med Phys 35:4404-9
Gur, David; Bandos, Andriy I; Rockette, Howard E (2008) Comparing areas under receiver operating characteristic curves: potential impact of the ""Last"" experimentally measured operating point. Radiology 247:12-5
Rockette, Howard E; Gur, David (2008) Selection of a rating scale in receiver operating characteristic studies: some remaining issues. Acad Radiol 15:245-8
Song, Tao; Bandos, Andriy I; Rockette, Howard E et al. (2008) On comparing methods for discriminating between actually negative and actually positive subjects with FROC type data. Med Phys 35:1547-58
Gur, David; Rockette, Howard E (2008) Performance assessments of diagnostic systems under the FROC paradigm: experimental, analytical, and results interpretation issues. Acad Radiol 15:1312-5
Gur, David; Rockette, Howard E; Bandos, Andriy I (2007) ""Binary"" and ""non-binary"" detection tasks: are current performance measures optimal? Acad Radiol 14:871-6
Gur, David; Bandos, Andriy I; Klym, Amy H et al. (2006) Reader variance in ROC studies--generalizability to reader population at high and low performance levels. Acad Radiol 13:1004-10

Showing the most recent 10 out of 15 publications