This project focuses on developing algorithms and datasets that can transform photometric reconstruction systems from hand-designed systems into learning-based systems that are optimized on real-world data. Photometric reconstruction systems derive cues from the perceived intensity of different locations on a surface. Shape-from-shading, where the surface is assumed to have a diffuse reflectance, is a well-known example of photometric reconstruction. This project produces the datasets and methods necessary to use machine learning techniques to build models for photometric reconstruction.
This learning-based approach enables systems to be optimized on real-world data so that they produce the most accurate results possible. In addition, this learning-based approach enables the development of more sophisticated methods with more parameters than typically used in hand-designed systems. The ability to find optimal parameters in an automated fashion can not only improve existing approaches, such as by incorporating image data more effectively, but can also enable the development of algorithms that push the boundaries of current systems. In particular, algorithms are developed for estimating the shape of objects without knowing the illumination or even trying to explicitly model it.
The power of the learning approach cannot be realized without data for training and testing. A major task in this work is the construction of a database of images and ground-truth 3D reconstructions of the objects in the images. The 3D models can be found using an example-based photometric stereo technique.
