PIs: Ramesh Raskar, MIT Media Lab Alyosha Molnar, Cornell ECE
Advanced imaging and display technology requires integrated, low cost systems able to efficiently capture and characterize light from 3-D scenes. In particular, a 3-D scene can be described by the collection of light rays it generates, called the light field. This research combines concepts from mask-based light-field imaging with angle sensitive pixels (ASPs). While mask-based light-field capture is much better understood mathematically, and masks are cheaper to manufacture and more easily modified on-the-fly, diffractive ASPs provide smaller, denser light field sensors, and provide naturally compressible outputs. This project combines the physics and signal processing of these approaches to enable optical imaging systems that capture more information than normal cameras while reducing the system's complexity. This work broadly impacts diverse applications spanning consumer imaging and displays, machine vision and automation, scientific/medical imaging and displays, robotic surgery, surveillance and remote sensing.
3-D images and video can be captured by measuring the combined spatial and angular distribution of light (the light field). This research combines two techniques for light-field capture: mask-based light-field imaging and diffractive angle sensitive pixels (ASPs). A critical element of this work is the development of a mathematical framework that maps between conventional geometric light fields and the diffractive optics upon which ASPs rely. A second element is constructing hybrid systems based on this mathematics, leveraging diffractive effects in mask design, and combining masks with ASPs in single light-field cameras. This work also combines formalisms in existing light field methods with knowledge about real 3-D scene statistics to develop optimal (in the sense of usability and compressibility) basis sets for sampling and encoding the light-field. All of these aspects combine to reduce the size, cost and complexity of light field cameras, while simultaneously enhancing their capabilities.
