The goal of this project is to develop robust algorithms for reconstructing the 3D shape and reflectance properties of real world scenes. The distinguishing feature of the proposed approach is the modeling of global light transport, taking into account both realistic reflection and global interreflection of light with surfaces in the unknown scene. Modeling realistic light transport is a open problem with major importance in computer vision, due to the fact that everyday materials reflect light in complex ways (e.g., wood, hair, velvet), and because the appearance of geometrically complex scenes is strongly affected by interreflection. This project seeks to model reflection in a very general setting, where the shape and the reflectance properties of the scene are both unknown and unconstrained. Modeling of interreflection will focus on diffuse scattering, and fully account for global light propagation through the scene. Toward this objective, a new computational framework is introduced that uses data-driven models of light transport that can be captured directly from photographs, and recovers scene structure without the need for complex simulations or optimizations of the underlying physical process. A key advantage of such data-driven models is that they work robustly in very general conditions.
The proposed work opens up new avenues in 3D shape sensing technology, a problem with widespread applications in robotics, visualization, mapping, aerial imaging, and virtual reality. The ability to capture material models on real objects will improve realism in computer graphics, and impacts applications such as entertainment, visualization, and the communication of visual media. The project will involve undergraduate and graduate research projects and the outcome will be a set of results and tools that will be broadly disseminated and incorporated into research projects and educational initiatives at the University of Washington.
