Machine learning (ML) and arti?cial intelligence have recently emerged as powerful techniques that can augment radiology interpretations and show promise for improving patient outcomes. One of the ways for ML to make a signi?cant impact on health care is in improving the evaluation of high-volume, low-cost exams for early signs of a wide variety of diseases.The routine chest x-ray is an ?opportunity for screening? for diseases, including cancer, chronic obstructive pulmonary disease (COPD), pneumonia and congestive heart failure. For instance, lung cancer is the most common cause of cancer death in the US, and is typically diagnosed at a higher stage than most other cancers leading to low survival rates. The National Lung Screening Trial reported that low dose computed tomography (LDCT) screening resulted in a 20% reduction in lung cancer mortality; however, few eli- gible people actually undergo LDCT screening. Meanwhile, chest x-rays continue to be the most common form of imaging worldwide. Improved detection from x-rays can direct patients to LDCT. COPD is another important disease that is often under-diagnosed. People with COPD are at increased risk of lung cancer and respiratory infections, or exacerbations, which are associated with higher morbidity and mortality. Furthermore, a chest x-ray may show poorly-de?ned regions of consolidation that are concerning for pneumonia. Medical attention is re- quired to treat an infection or evaluate for other cause. More generally, methods to detect disease on chest x-rays can be extended to cardiomegaly, pulmonary edema and pleural effusions which are seen in congestive heart failure. Improved detection can direct patients to medical care. Convolutional neural networks (CNN), a highly successful ML model, can be applied to chest x-ray images. However, few annotated medical datasets exist that are suf?ciently large to train CNNs. Furthermore, it has been shown that bounding boxes used to localize disease can be incorporated into the training of CNNs and signi?cantly increase their accuracy. Unfortunately, medical datasets with such localized annotations are even rarer and are very limited in the number of cases due to the time-consuming process of creating bounding boxes by radiologists. We propose an innovative integrated approach using eye tracking, speech recording and novel vision and language models to create localized annota- tions in a manner that is non-intrusive to the work?ow of the radiologist. The novelty of our approach is in the use of eye tracking during routine radiological reading. The challenge is to overcome the relatively ambiguous nature of eye tracking information compared to bounding boxes which provide de?nitive information about abnormalities. To address this challenge, we will also design new CNN architectures and learning algorithms that can use eye tracking and additional information such as pupil dilation and ?xation duration. The proposed methodology can easily scale up to create very large datasets without generating additional workload for radiologists. Furthermore, deployed in the reading room, it could provide a continuous stream of annotated images to expand training sets.
Machine learning and arti?cial intelligence hold the potential to improve patient outcomes by enhancing the eval- uation of low cost, routine chest x-rays for early imaging abnormalities associated with diseases such as lung cancer, chronic obstructive pulmonary disease and pneumonia and making recommendations for follow-up care. To realize their full potential, machine learning algorithms require training on very large datasets with localized an- notations for abnormalities. We propose a novel data collection methodology using eye tracking that is scalable, non-intrusive to the work?ow of the radiologist, and can generate very large datasets with localized annotations for the development of novel machine learning algorithms to address signi?cant medical problems.
