Imaging technology plays a critical role in advancing science. However, as science continues to push the boundaries of knowledge, traditional imaging sensors are reaching their limits. For example, traditional telescopes cannot be constructed large enough to resolve a black hole’s shadow; traditional microscopes are not able to see transparent cells, and traditional cameras cannot be used to study the inner core of a cloud due to scattering. Breaking these fundamental limits has only been possible through the emergence of computational cameras, which replace optics with computational algorithms; this new paradigm shift has enabled image formation processes that were previously infeasible for conventional optical imaging. The full potential of computational cameras is far from being realized; thus far they have primarily been identified and developed though human ingenuity. Consequently, computational imaging pipelines are often significantly under-optimized, and there is no doubt that many such “cameras†have yet to even be identified. Developing the next generation of computational cameras requires a fundamental shift away from relying on human intuition and overly simplified models in the design of imaging pipelines. This project aims to develop modern learning-based approaches to jointly optimize sensor and algorithm designs in computational camera pipelines in order to automatically discover new imaging strategies.
The objective of this project is to develop a data-driven, generalizable learning framework that solves for a jointly optimized sensor design and reconstruction algorithm for computational imaging pipelines. The generalizable co-design framework will be developed to easily incorporate domain knowledge and respect physical constraints. In collaboration with domain-experts, the investigator will study the application of this framework to problems ranging from astronomical imaging to seismic imaging. The investigator will pursue fundamental work in four areas: 1) single-shot probabilistic co-design to optimize sensor design jointly with reconstruction methods, 2) online sequential probabilistic co-design for optimizing the next sensor measurement conditioned on previous measurements for a particular target, 3) co-design with a stochastically evolving target, and 4) co-design with a mismatched forward model. The investigator will make use of emerging computational techniques and machinery in machine learning, signal processing, optimization, applied math, and controls to efficiently co-optimize the computational imaging pipeline. This research will transform the way novel imaging pipelines are identified and developed, and will result in the development of new methods that will impact a wide array of important imaging problems, including astronomical, medical, seismic, and microscopic imaging.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
