The vast majority of computer vision techniques continue to be predicated on restrictive assumptions about reflectance, and their ability to extract meaningful information from images of complex reflecting scenes remains limited. The proposed research activity works toward a framework for the analysis of complex reflecting scenes through the decomposition of reflectance. According to this approach, a reduced representation of an image is obtained on a point-by-point basis by its decomposition into simpler constituents; and this representation provides access to scene information (e.g., shape, illumination, material properties) that would be otherwise inaccessible. The distinguishing feature of this research is that instead of directly recovering complete estimates of reflection components, preliminary reduced representations will be sought, which isolate diffuse reflection effects in some usable form. This makes the problem tractable and enables the development of a robust and general framework for the analysis of complex reflecting scenes. The specific goals of the proposed research are: (1) Reduced representations that isolate the much simpler diffuse reflection effects in scenes with variable and complex illumination; and (2) Methods for dense, region-based tracking that apply the reduced representations.
Improvements in visual tracking will facilitate robust navigation systems and human-computer interfaces. Enhanced recognition systems will benefit systems for visual inspection, surveillance and homeland security (e.g., face recognition). Improvements in 3D reconstruction techniques will enhance a system's ability to learn appearance models of objects from their images, thereby enabling the system to predict the appearance of these objects in novel environments. Under this grant, the PI will also develop curricula for two undergraduate courses (at the freshmen and senior levels) at Harvard University. The freshman-level course will be designed to attract students from underrepresented groups into engineering and computer science by exposing them to intuitive and exciting computational aspects of visual understanding.
