Since the inception of the paradigm in 1961 the analysis of FROC (free-response receiver operating characteristic) data (i.e., mark-rating pairs) has remained an unsolved problem in imaging science. Since the radiologist must search patient images for localized lesions, intrinsically FROC data is generated in virtually all imaging studies in radiology. Currently there is great interest in evaluating different computer aided detection (CAD) algorithms for lung cancer screening with helical CT scans. This task also involves search and depending on the CAD in use there are approximately 4 to 20 suspicious areas reported per patient While in the past the receiver operating characteristic (ROC) paradigm has been the standard method for the evaluation of diagnostic imaging systems, its limitation to one report per patient is now being increasingly felt. This is a proposal to develop the science and perform the requisite testing of the FROC paradigm by accomplishing the following aims. (1) Develop a statistical model for FROC data that includes the effect of search and other relevant perceptual factors and which reduces to the standard ROC model in non-search tasks. (2) Use the model to validate and evaluate currently proposed methods for analyzing location and ratings data, and in particular determine their statistical power. (3) Develop an algorithm to determine the parameters of the model from clinical FROC data and to make statistical inferences regarding the modalities being tested; verify that the estimated search parameters correlate with values inferred from eye-movement recordings. (4) Provide practical and validated software to the user community that will offer more statistical power and therefore require fewer resources of cases and readers. The significance of the proposed work is that on the scale of previous developments, the potential gain in statistical power by the FROC method is very large. Also, the explicit modeling of search and the ability to estimate search related parameters from observer data should open up new areas of vision research that all researchers can exploit.

Agency
National Institute of Health (NIH)
Institute
National Institute of Biomedical Imaging and Bioengineering (NIBIB)
Type
Research Project (R01)
Project #
5R01EB005243-04
Application #
7425830
Study Section
Biomedical Imaging Technology Study Section (BMIT)
Program Officer
Cohen, Zohara
Project Start
2005-08-01
Project End
2009-12-31
Budget Start
2008-06-01
Budget End
2009-12-31
Support Year
4
Fiscal Year
2008
Total Cost
$315,322
Indirect Cost
Name
University of Pittsburgh
Department
Radiation-Diagnostic/Oncology
Type
Schools of Medicine
DUNS #
004514360
City
Pittsburgh
State
PA
Country
United States
Zip Code
15213
Dobbins 3rd, James T; McAdams, H Page; Sabol, John M et al. (2017) Multi-Institutional Evaluation of Digital Tomosynthesis, Dual-Energy Radiography, and Conventional Chest Radiography for the Detection and Management of Pulmonary Nodules. Radiology 282:236-250
Thompson, J D; Chakraborty, D P; Szczepura, K et al. (2016) Effect of reconstruction methods and x-ray tube current-time product on nodule detection in an anthropomorphic thorax phantom: A crossed-modality JAFROC observer study. Med Phys 43:1265-74
Mackenzie, Alistair; Warren, Lucy M; Wallis, Matthew G et al. (2016) Breast cancer detection rates using four different types of mammography detectors. Eur Radiol 26:874-83
Mackenzie, Alistair; Warren, Lucy M; Wallis, Matthew G et al. (2016) The relationship between cancer detection in mammography and image quality measurements. Phys Med 32:568-74
Warren, Lucy M; Given-Wilson, Rosalind M; Wallis, Matthew G et al. (2014) The effect of image processing on the detection of cancers in digital mammography. AJR Am J Roentgenol 203:387-93
Soh, B P; Lee, W B; McEntee, M F et al. (2014) Mammography test sets: reading location and prior images do not affect group performance. Clin Radiol 69:397-402
Thompson, John D; Hogg, Peter; Manning, David J et al. (2014) A free-response evaluation determining value in the computed tomography attenuation correction image for revealing pulmonary incidental findings: a phantom study. Acad Radiol 21:538-45
Chakraborty, Dev P (2013) A brief history of free-response receiver operating characteristic paradigm data analysis. Acad Radiol 20:915-9
Haygood, T M; Ryan, J; Brennan, P C et al. (2013) On the choice of acceptance radius in free-response observer performance studies. Br J Radiol 86:42313554
Soh, BaoLin P; Lee, Warwick; McEntee, Mark F et al. (2013) Screening mammography: test set data can reasonably describe actual clinical reporting. Radiology 268:46-53

Showing the most recent 10 out of 38 publications