Eye-tracking methods are fundamental for the study of medical image perception and central to discoveries about why radiologist interpretative failures occur. However, analysis of eye-tracking data has been rather primitive and ignores potentially valuable information. We propose a new approach to the analysis of eye-tracking data based on a recently developed model for the analysis of receiver operating characteristic (ROC) data, free-response operating characteristic (FROC) mark-rating data, and location- specific ROC (LROC) data. Based on Chakraborty's model, we propose a method that integrates the analysis of true and false positive characterizations of eye position data and uses the degree of suspicion of radiologist indicated suspicious regions. The inclusion of the FROC performance data is expected to yield better understanding of image perception than is possible via eye-tracking alone. The integrated analysis is enabled by two recent developments: the Chakraborty search model and a method for estimating its parameters. The parameters of the model correspond to physical quantities that are measured in eye-tracking studies. The project consists of quantifying these correspondences, providing a demonstration of integrated analysis, and showing its advantages over eye-tracking alone. Eye-tracking and FROC studies have so far proceeded along independent tracks, one to understand image perception and the other to measure performance. This project shows how a combined approach can yield a more powerful tool for analyzing eye-tracking data and understanding image perception. With better understanding of image perception based on improved analysis, we will be better able to improve diagnostic performance. Applications of this method include radiologist training and improved CAD algorithms. The rich dataset of simultaneously acquired FROC and eye-position data, and analysis software will be made freely available at the close of our project.

Public Health Relevance

Eye-tracking apparatus measures where radiologists look. This information is fundamental to understand medical image perception and central to learning why radiologist interpretative failures occur. However, analysis of such data has been primitive and ignores valuable information. We propose a new approach to the analysis of eye-tracking data. With better understanding of image perception we will be better able to improve radiologist performance and reduce interpretive errors.

Agency
National Institute of Health (NIH)
Institute
National Institute of Biomedical Imaging and Bioengineering (NIBIB)
Type
Research Project (R01)
Project #
5R01EB008688-03
Application #
7799280
Study Section
Special Emphasis Panel (ZRG1-SBIB-P (02))
Program Officer
Cohen, Zohara
Project Start
2008-07-01
Project End
2012-03-31
Budget Start
2010-04-01
Budget End
2011-03-31
Support Year
3
Fiscal Year
2010
Total Cost
$350,608
Indirect Cost
Name
University of Pittsburgh
Department
Radiation-Diagnostic/Oncology
Type
Schools of Medicine
DUNS #
004514360
City
Pittsburgh
State
PA
Country
United States
Zip Code
15213
Soh, B P; Lee, W B; McEntee, M F et al. (2014) Mammography test sets: reading location and prior images do not affect group performance. Clin Radiol 69:397-402
Thompson, John D; Hogg, Peter; Manning, David J et al. (2014) A free-response evaluation determining value in the computed tomography attenuation correction image for revealing pulmonary incidental findings: a phantom study. Acad Radiol 21:538-45
Warren, Lucy M; Given-Wilson, Rosalind M; Wallis, Matthew G et al. (2014) The effect of image processing on the detection of cancers in digital mammography. AJR Am J Roentgenol 203:387-93
Haygood, T M; Ryan, J; Brennan, P C et al. (2013) On the choice of acceptance radius in free-response observer performance studies. Br J Radiol 86:42313554
Chakraborty, Dev P (2013) A brief history of free-response receiver operating characteristic paradigm data analysis. Acad Radiol 20:915-9
Chakraborty, D P; Haygood, T M; Ryan, J et al. (2012) Quantifying the clinical relevance of a laboratory observer performance paradigm. Br J Radiol 85:1287-302
Chakraborty, Dev P; Yoon, Hong-Jun; Mello-Thoms, Claudia (2012) Application of threshold-bias independent analysis to eye-tracking and FROC data. Acad Radiol 19:1474-83
Svahn, T M; Chakraborty, D P; Ikeda, D et al. (2012) Breast tomosynthesis and digital mammography: a comparison of diagnostic accuracy. Br J Radiol 85:e1074-82
Chakraborty, Dev P (2012) Measuring agreement between rating interpretations and binary clinical interpretations of images: a simulation study of methods for quantifying the clinical relevance of an observer performance paradigm. Phys Med Biol 57:2873-904
Zanca, Federica; Hillis, Stephen L; Claus, Filip et al. (2012) Correlation of free-response and receiver-operating-characteristic area-under-the-curve estimates: results from independently conducted FROC∕ROC studies in mammography. Med Phys 39:5917-29

Showing the most recent 10 out of 23 publications