Animals see in many different ways. This is especially true in invertebrates and small reptiles, where evolution has produced vision systems that are particularly small, specialized, and efficient. Mimicking this specialized efficiency requires what engineers call "computational sensors," meaning sensors whose optics and algorithms work synergistically to improve accuracy while reducing sensor size and power requirements. This project will explore the creation of a new generation of visual sensors that use metasurfaces for their optics. Metasurfaces are ultrathin translucent sheets patterned with nanoscale structures that are tailored to interact with light in specific ways. In addition to being smaller and lighter than conventional optics, they allow unprecedented control of light's various properties, including phase, spectrum, and polarization. The project will create mathematical tools and software tools that allow engineers to create new visual sensors using metasurfaces. It will also use these tools to create prototypes of small visual sensors that mimick the depth-sensing capabilities of jumping spiders, which are remarkably small and efficient.
The project will increase the variety of optics that can be considered during sensor design. It will also enable new optics to be learned automatically instead of having to be manually designed. Both of these will follow from the unified consideration of optics and computation, which this project will accomplish by treating metasurfaces as optical processing layers that precede the usual computational layers of a feed-forward network. The project will create mathematical tools that allow metasurface shapes to be optimized by back-propagation, in conjunction with optimizing the parameters and hyper-parameters of post-capture computations. At the systems level, the project will produce prototypes of passive snapshot depth sensors and snapshot RGBD sensors that are very small and power efficient. These sensors will use thin, millimeter-diameter metasurfaces and minimal amounts of post-capture computation.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
