This project targets the development of monocentric camera systems for high-resolution, wide field-of-view (FOV) light field imaging in small device form factors. Building on the benefits of recently-developed monocentric optics - ultra-high resolution, small physical footprint, low weight, and high light collection - monocentric light field imagers provide a transformative platform for a range of future experiential imaging and computing applications. In particular, light field-enabled monocentric optics allow for spatially-varying digital focus for complex and wide FOV scenes, 3D imaging capabilities, stereo view synthesis, and imaging through partial occluders. As opposed to any existing technology, monocentric light field imagers enable immersive content for emerging head-mounted displays with support for focus cues to be captured with low-cost, mobile devices. A range of computer vision algorithms directly benefit from the targeted computational imaging platform, including 4D feature detection, localization and mapping, segmentation, recognition, tracking, depth estimation, matting, object removal, and hole filling. The developed monocentric light field imaging system provides benefits for society at large; the enabled 3D image capture and editing capabilities offered in a small device form factor could profoundly impact future means of inter-personal digital communication, remote collaboration and education as well as remote operation of vehicles. Newly-developed computer vision algorithms are beneficial for navigation of autonomous vehicles. Live content for a range of applications can be easily recorded and edited, for example for simulation, training, phobia treatment, and cultural heritage. Light field optics and algorithm design will be tightly integrated into the syllabus of multiple graduate-level courses at Stanford and UCSD and made available to industry professionals via online learning platforms.
This research investigates a viable solution for these challenges and provides a next-generation computational imaging platform. Leveraging the expertise of PIs from University of California San Diego and Stanford University, this project aims at (i) designing and fabricating a wide field of view light field imager via monocentric optics, conformal microlenses, and fiber coupling, (ii) developing end-to-end computational imaging pipelines, from coded capture to display on emerging head mounted displays, and (iii) evaluating computer vision and scene understanding algorithms, including feature detection, localization, mapping, segmentation, classification, tracking, matting, classification, and object removal. The research question driving this project is the quest for a small, computational imaging system that is flexible enough to unlock a range of visual and experiential computing applications that cannot be easily provided by cameras available today. Monocentric optics offer great benefits for such applications: wide field of view, high resolution, high light collection, and a small form factor. Yet, future visual computing applications require even more functionality: 3D imaging, adaptive digital focus over a large FOV, compatibility with emerging virtual and augmented reality displays, enhanced image editing modes, such as object segmentation, removal, insertion, localization, and more.
