The predominant approach to computer-aided diagnosis (CAD) in medical imaging has been to use automated image analysis to serve as a """"""""second reader,"""""""" with the aim of improving radiologists'diagnostic performance. CAD techniques traditionally aim to highlight suspicious lesions (called CADe) and/or estimate diagnostic variables, such as probability of malignancy (called CADx). We have been developing and evaluating a different approach to CAD, in which the radiologist will be assisted by a content-based search engine that will automatically identify and display examples of lesions, with known pathology, that are similar to the lesion being evaluated (referred to as the query). This will involve searching a large database for the images that are most similar to the query, based on image features that are automatically extracted by the software. The philosophy of this approach is to help inform the radiologist's diagnosis in difficult cases by presenting relevant information from past cases. The retrieved example lesions will allow the radiologist to explicitly compare known cases to the unknown case. A key advantage of the proposed retrieval approach to CAD is that it leaves decision-making entirely in the hands of the radiologist, unlike CADx, which acts as a supplemental decision maker. In our approach, we aim to tackle the key challenge of image retrieval, which is to develop a meaningful computerized measure of the similarity (relevance) of a patient's images to other images in the database. Departing from typical approaches based on numerical distance measures, we have proposed that the most useful measure of similarity is one that is designed specifically to match that perceived by the radiologist. We postulate that the radiologist's notion of similarity is some complicated unknown function of the images, and use advanced machine-learning algorithms to learn this function from similarity scores collected from radiologists in reader studies. Under R21 funding, we successfully demonstrated the feasibility and good performance of our approach in small data sets. The purpose of this proposed R01 project is to follow up the R21 project with a significantly larger scale effort in order to bring this approach to fruition, which will lead to a suite of retrieval-based CAD tools. We will develop the following unique components toward a clinical diagnostic aid: 1) instead of using indexing terms or simple distance measures to identify relevant images in the database, the system will use a similarity measure specifically trained to match radiologists'notion of relevance, as inferred from data obtained in an observer study;2) in addition to presenting the retrieved cases to the radiologist, the system will use them to boost a CADx classifier to improve its classification accuracy on the query lesion;3) the system will have the new capability of automatically building a large reference library by extracting known cases from a hospital PACS, thereby maximizing the benefit by retrieving more-similar cases;and 4) the system will be augmented with a highly interactive interface, which will include new tools for automatically adapting the similarity measure according to users'preferences, and for effectively presenting retrieved results. All of these components are novel and important to ultimate success of this kind of diagnostic aid. The project will include a preliminary demonstration using the Hospital Information System at the University of Chicago Hospitals, and will include preliminary evaluation studies to determine the effect of the system on radiologists'diagnostic performance.

Public Health Relevance

This project will focus on development of a suite of supporting tools to facilitate the interpretation of images in radiology by mining similar cases from a database. The proposed system will make available to the radiologist through an intuitive interface a broad selection of relevant past cases to the one being diagnosed, along with an improved measure of its malignancy that is boosted by using retrieved cases, from which the radiologist can draw his or her own conclusions. We hypothesize that by providing such case-based evidence it will help radiologists in their decision-making process, particularly in diagnosis of difficult cases.

National Institute of Health (NIH)
National Institute of Biomedical Imaging and Bioengineering (NIBIB)
Research Project (R01)
Project #
Application #
Study Section
Biomedical Imaging Technology Study Section (BMIT)
Program Officer
Pai, Vinay Manjunath
Project Start
Project End
Budget Start
Budget End
Support Year
Fiscal Year
Total Cost
Indirect Cost
Illinois Institute of Technology
Engineering (All Types)
Schools of Engineering
United States
Zip Code
Sainz de Cea, MarĂ­a V; Nishikawa, Robert M; Yang, Yongyi (2018) Locally adaptive decision in detection of clustered microcalcifications in mammograms. Phys Med Biol 63:045014
Wang, Juan; Yang, Yongyi (2018) A context-sensitive deep learning approach for microcalcification detection in mammograms. Pattern Recognit 78:12-22
Sainz de Cea, Maria V; Nishikawa, Robert M; Yang, Yongyi (2017) Estimating the Accuracy Level Among Individual Detections in Clustered Microcalcifications. IEEE Trans Med Imaging 36:1162-1171
Wang, Juan; Nishikawa, Robert M; Yang, Yongyi (2017) Global detection approach for clustered microcalcifications in mammograms using a deep learning network. J Med Imaging (Bellingham) 4:024501
Wang, Juan; Nishikawa, Robert M; Yang, Yongyi (2016) Improving the accuracy in detection of clustered microcalcifications with a context-sensitive classification model. Med Phys 43:159
Wang, Juan; Jing, Hao; Wernick, Miles N et al. (2014) Analysis of perceived similarity between pairs of microcalcification clusters in mammograms. Med Phys 41:051904
Jing, Hao; Yang, Yongyi; Nishikawa, Robert M (2012) Retrieval boosted computer-aided diagnosis of clustered microcalcifications for breast cancer. Med Phys 39:676-85
Jing, Hao; Yang, Yongyi; Nishikawa, Robert M (2012) Regularization in retrieval-driven classification of clustered microcalcifications for breast cancer. Int J Biomed Imaging 2012:463408
Jing, Hao; Yang, Yongyi; Wernick, Miles N et al. (2012) A comparison study of image features between FFDM and film mammogram images. Med Phys 39:4386-94
Jing, Hao; Yang, Yongyi; Nishikawa, Robert M (2011) Detection of clustered microcalcifications using spatial point process modeling. Phys Med Biol 56:1-17

Showing the most recent 10 out of 12 publications