Image segmentation, which extracts objects of interest from given images, is a fundamental computer vision task. This project develops novel image segmentation methodology combining classic mathematical foundations and modern deep neural networks. In particular, the developed methodology will achieve high quality in segmenting fine-scale object instances, as well as their topology. Correct segmentation of fine-details and topology such as connectivity between parts is critical for downstream analysis such as reasoning about affordance of objects - what actions can be made on them - and biomedical image analysis. This project not only bridges the gap between principled mathematical theory and the practical deep image segmentation framework, but also trains the next generation of researchers and educators. Through a carefully designed integrated educational and outreach plan, the principal investigators will engage undergraduate students, high school students, women, and other underrepresented students in the research activities.
This project studies deep variational relaxations of segmentation problems, namely, consider the segmentation task as a continuous valued prediction problem and employ variational functionals as training loss functions for deep neural networks. The introduction of deep learning allows highly nonlinear functions to be estimated and greatly improves the capability of variational approaches such as the Mumford-Shah functional and the persistent homology, in segmenting instances with sharp boundaries and with correct topology. Applications on robotic affordance and influence prediction and medical imaging will improve state-of-the-arts in those areas. The resulting techniques and software will be validated on image segmentation, affordance and medical imaging datasets, in order to provide quantitative assessments of the proposed approaches.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
