Central to the imaging process is the interaction of light with the objects in the scene. Remarkable progress has been made over the past several hundred years on solving inference problems (such as detection, classification, or estimation) for a large class of objects constructed from simple materials with Lambertian reflectance. Such objects scatter light such that the apparent brightness is invariant to the observer's view angle. Unfortunately, real world objects are made of considerably more complex materials that cannot be characterized in terms of such an isotropic reflectance. While humans are able to effortlessly reason about complex materials, today's image analysis and processing algorithms fail miserably. The reason is that complex, non-Lambertian materials can be characterized only by higher-dimensional functions that are relatively poorly understood and even more poorly modeled.
This research is developing new ways to model, capture, and process the rich reflectance patterns of complex materials. The key tool is the object's plenoptic transport function, which describes the transformation of the incident to the irradiated light due to the properties of the material. In full generality, the plenoptic transport function is 14-dimensional; hence, a fundamental complication for sensing, analysis, and processing systems for complex materials is the dimensionality gap between the high dimensional plenoptic function and the ability of most conventional sensors (cameras) to acquire at best 2D or 3D image projections. The tools and techniques under development include sparse and manifold models (to bridge the dimensionality gap), geometric features (to mitigate the presence of environmental illumination and other nuisance parameters), and new sensor designs (to most efficiently acquire plenoptic information from natural scenes).
