A perspective camera captures the spatial relationships of objects in a scene as they would appear from a single viewpoint. In contrast, a multi-perspective camera combines what is seen from several viewpoints into a single image and provides a potentially advantageous imaging system for understanding the structure of observed scenes. However, most existing computer vision and graphics algorithms are not directly applicable to multi-perspective cameras. The goal of this project is to develop a complete framework to characterize and design new multi-perspective cameras and displays and to use these systems for computer vision and graphics applications.
On the vision front, the PI explores two types of real multi-perspective cameras: the first extends existing catadioptric mirrors and the second uses specially-shaped apertures. To categorize these cameras, a ray geometry analysis is applied to identify important ray structures and study their implications on image distortions. The PI also investigates several new 3D reconstruction methods, such as epsilon stereo matching, ray-curvature analysis, and curvature-from-distortions, for a broad class of multi-perspective cameras. On the graphics front, the PI explores practical multi-perspective display architectures by combining a consumer projector with specially-shaped mirrors/lenses. Such displays can offer an unprecedented level of flexibility in terms of aspect ratio, size, field of view, etc. The PI also investigates real-time multi-perspective rasterization and frustum culling algorithms by modifying the traditional rendering pipeline.
The project contributes to education by involving undergraduate and graduate students in self-contained projects with both theory and application components, and attracting under-represented students through the pre-engineering program between the University of Delaware and the Delaware State University.
