This project endeavors to develop novel computational representations for appearance modeling of physical materials for realistic computer graphics and machine vision applications by investigating spherical moments for modeling material appearance. Beyond the development of a novel theoretical framework for appearance modeling, the project involves evaluation of the generalization of the proposed appearance representation in various settings using software simulations as well using ground truth data available from various measurements. The project identifies three fundamental goals for such an appearance representation: application to general forward (reflectance modeling) and inverse (reflectance estimation) problems, appearance modeling in uncontrolled (lighting and viewpoint) settings, and validation and appearance classification for scene analysis applications. In particular, the analysis of higher order statistics for novel compact representations for forward simulation and inverse rendering problems are investigated. Additionally, appearance modeling from sparse input data acquired under general conditions of semi-controlled or uncontrolled lighting is investigated within this framework. Also investigated is the classification of material appearance from sparse measurements based on such spherical statistics. This research has far-reaching impact beyond computer graphics in many fields such as architecture, engineering, science, fine art and entertainment. Besides providing new insights into appearance modeling, the developed theory allows rapid measurements with greater ease, making the results more accessible to other researchers and practitioners in the field. Besides mentoring a graduate student, the findings of the research are planned to be integrated into a graduate level computer science course offering and through the creation of internship opportunities for undergraduate students.
Digitally reproducing the appearance of real-world subject is a challenging problem. Access to an accurate description of a subject’s appearance enables us to visualize the subject from varying viewpoints and under varying lighting conditions. One of the most successful strategies for obtaining the appearance of physical objects relies on probing the subject’s appearance under various controlled lighting conditions. This project investigated a novel family of such controlled lighting conditions, based on higher order spherical statistics, to efficiently acquire various aspects of real-world appearance. Many real-world materials exhibit some form of translucency, creating a characteristic ‘soft’ appearance. An example of such a material is human skin. The appearance of translucent materials can be characterized by quantifying how much the material scatters light and how much it absorbs. We established, both theoretically as well as practically, the relation between surface curvature (which can also be efficiently estimated using the same type of observations) and the scattering and absorption parameters. Additionally, during the course of this project we developed the concept of a Stokes Reflectance field related to the polarization property of light and studied how this can aid in estimating how materials reflect incident lighting. In particular, we presented the first method that can estimate spatially varying reflectance parameters as well as index of refraction of opaque materials. In follow up work we employed the Stokes Reflectance field to estimate surface shape detail under uncontrolled outdoor lighting conditions. Recently, we showed how higher order spherical reflectance statistics enable, for the first time, a unified and practical approach for measuring both shape and appearance of both shiny as well as matte objects. This last contribution non-trivially ties together all the individual scientific developments over the course of this proposal. Our research has advanced the state-of-the-art in both appearance and geometry acquisition in the computer graphics and computer vision field, and has resulted in several scientific journal publications. Moreover, this project has supported the development of multiple graduate students, summer interns, and undergraduate students. Its results have been integrated in undergraduate and graduate courses at the participating institutes. Our methods are currently utilized in the entertainment industry and e-commerce. The developed theories and methods have applications in fields as diverse as security, industrial quality control, and medical sciences.
