The goals of the Deep Learning TRD of the Advanced Technologies for the National Center for Image-Guided Therapy (AT-NCIGT) are to investigate revolutionary advances in deep learning (DL) in the context of image-guided therapy (IGT) of brain, prostate, and lung cancer, and to develop tools that can be used by the broader IGT research community. The general theme of our research addresses difficulties associated with creation of training data, which is a significant impediment to the application of DL to medical images. While DL has had many successes in image-based classification or segmentation tasks, the methods used are fully supervised, i.e., very large amounts of accurately annotated training data are needed to achieve best performance. Currently, expert annotation is expensive and laborious in the case of medical images because accurate segmentation of 3D structures requires manual or semi-automatic labeling of thousands of voxels per image. Concurrently, large unlabeled or weakly-labeled data sets are becoming available. For example, the PACS system of a large hospital might contain tens of millions of scans, but determining accurate disease labels is difficult. There are currently two promising DL approaches that can be used to address this problem, weakly-supervised learning (where some labels are absent or otherwise imperfect) and transfer learning (which leverages labeled data sets that are in some ways similar). The current situation is further exacerbated by a lack of machine readable metadata, and of methods and tools to support curation of the imaging (e.g., Magnetic Resonance Imaging (MRI)) and clinical data, alongside annotations and analysis results within a single data model. The latter leads to fragmentation of data, and non-standard and heterogeneous metadata. We address these problems by 1) Developing new information theoretic technology for weakly-supervised deep learning, 2) developing novel training strategies for deep learning for cancer characterization for transperineal in-bore MRI-guided prostate biopsy, and 3) developing an infrastructure for curating imaging data for deep learning. The results of this TRD will be DL algorithms, the resulting models, and tools for annotation and organizing machine-readable metadata that are designed to enable IGT cancer research for the prostate, the lung, and the brain applications.
