The research involves measuring imaging system performance in tasks such as detecting breast cancer. Receiver operating characteristic (ROC) methodology, the current gold-standard, uses patient-level information that a woman has suspected breast cancer. The location-specific free-response ROC (FROC) method uses additional location-level information in the radiologist's report, e.g., the cancer is in the left breast and is present at a particular location. Progress during the funded period has resulted in a novel perceptually-based FROC model and data simulator and several validated methods for analyzing data which are applicable to human observers and computer aided detection (CAD) algorithms. Papers using the PI's ideas and software are being presented in increasing numbers at conferences and in journals, and his work has generated healthy debate. The overall goal of the competing renewal project is to continue advancing the state-of-the-art in this field by addressing a number of limitations of current methods.
Specific Aim 1 : The figure-of-merit (FOM) is a critical determinant of statistical power and clinical relevance but all current FOMs are lesion-based and cases with more lesions contribute more to the FOM than cases with fewer lesions, and clinically less important lesions contribute equally as more important ones;we will develop novel case-based FOMs that overcome these limitations.
Specific Aim 2 : A realistic simulator yields confidence in methodology validation using that simulator. We will extend the current simulator by incorporating more realistic correlation effects and we will develop methodology to calibrate the simulator to real datasets thereby allowing the methodology developer to tune the simulator to specific applications. The simulator will be used to validate the different methods of analysis developed in Aim 1.
Specific Aim 3 : We will address several practical issues with current FROC methodology: arbitrariness of the proximity criterion, i.e., how close a mark must be to a lesion in order to credit the observer for a true detection;lack of sample-size estimation methodology for planning prospective studies;and lack of methods for analyzing clinically realistic data acquisition scenarios such as multiple views and breasts and multiple lesion types per case.
Specific Aim 4 : We will validate the methodology using independently acquired ROC, FROC and outcome-data in mammography. Outcome is defined as GOOD for normal cases returned to screening or abnormal cases sent to biopsy and BAD otherwise. We will test the hypothesis that FROC better correlates with outcome and yields greater statistical power than ROC. The significance is that the field is increasingly moving towards location-specific analyses, because of its intrinsic appeal and clinical realism, therefore methodology capable of analyzing the complex data, well outside the scope of the current gold-standard, is urgently needed. Patients benefit from better designed and optimized equipment leading to early diagnosis and treatment of cancers. Health care benefits because more efficient and cost-effective studies become possible which could serve as surrogates for expensive clinical trials.

Public Health Relevance

The research involves measuring imaging system performance in tasks such as detecting breast cancer where the current gold-standard uses information that a woman is suspected to have breast cancer, but the proposed new method uses additional information, e.g., the suspected cancer is in the left breast at a particular location, thereby yielding a more precise and clinically relevant measurement of performance. Patients benefit from better designed and optimized equipment leading to early diagnosis and treatment of cancers. Health care benefits because more efficient and cost-effective studies become possible which could serve as surrogates for expensive clinical trials.

Agency
National Institute of Health (NIH)
Institute
National Institute of Biomedical Imaging and Bioengineering (NIBIB)
Type
Research Project (R01)
Project #
5R01EB005243-08
Application #
8426180
Study Section
Biomedical Imaging Technology Study Section (BMIT)
Program Officer
Conroy, Richard
Project Start
2005-08-01
Project End
2014-12-31
Budget Start
2013-01-01
Budget End
2014-12-31
Support Year
8
Fiscal Year
2013
Total Cost
$380,999
Indirect Cost
$129,514
Name
University of Pittsburgh
Department
Radiation-Diagnostic/Oncology
Type
Schools of Medicine
DUNS #
004514360
City
Pittsburgh
State
PA
Country
United States
Zip Code
15213
Mackenzie, Alistair; Warren, Lucy M; Wallis, Matthew G et al. (2016) The relationship between cancer detection in mammography and image quality measurements. Phys Med 32:568-74
Mackenzie, Alistair; Warren, Lucy M; Wallis, Matthew G et al. (2016) Breast cancer detection rates using four different types of mammography detectors. Eur Radiol 26:874-83
Thompson, J D; Chakraborty, D P; Szczepura, K et al. (2016) Effect of reconstruction methods and x-ray tube current-time product on nodule detection in an anthropomorphic thorax phantom: A crossed-modality JAFROC observer study. Med Phys 43:1265-74
Soh, B P; Lee, W B; McEntee, M F et al. (2014) Mammography test sets: reading location and prior images do not affect group performance. Clin Radiol 69:397-402
Thompson, John D; Hogg, Peter; Manning, David J et al. (2014) A free-response evaluation determining value in the computed tomography attenuation correction image for revealing pulmonary incidental findings: a phantom study. Acad Radiol 21:538-45
Warren, Lucy M; Given-Wilson, Rosalind M; Wallis, Matthew G et al. (2014) The effect of image processing on the detection of cancers in digital mammography. AJR Am J Roentgenol 203:387-93
Haygood, T M; Ryan, J; Brennan, P C et al. (2013) On the choice of acceptance radius in free-response observer performance studies. Br J Radiol 86:42313554
Chakraborty, Dev P (2013) A brief history of free-response receiver operating characteristic paradigm data analysis. Acad Radiol 20:915-9
Chakraborty, D P; Haygood, T M; Ryan, J et al. (2012) Quantifying the clinical relevance of a laboratory observer performance paradigm. Br J Radiol 85:1287-302
Chakraborty, Dev P; Yoon, Hong-Jun; Mello-Thoms, Claudia (2012) Application of threshold-bias independent analysis to eye-tracking and FROC data. Acad Radiol 19:1474-83

Showing the most recent 10 out of 36 publications