Photography has an enormous impact on society -- it is our primary visual history and a medium for storytelling, entertainment, and art. But our visual world is extraordinarily complex which makes it difficult for computer vision to understand photos and for computer graphics to synthesize visual content. However, the emergence of Internet-scale photo collections in recent years enables new research directions. We use scene-based representations to leverage Internet-scale data. Scenes (places or environments) are the context in which all other visual phenomena exist and it seems possible to brute-force the space of scenes -- with millions of scenes, we find qualitatively similar scenes and create massively data-driven algorithms with capabilities that are complementary to typical bottom-up graphics and vision pipelines. The underlying principle of this study is that joint investigations of scene representations and large image databases will advance the state-of-the-art in graphics and vision.
First, we are investigating detail synthesis tasks which alleviate camera shake, motion blur, defocus, atmospheric scattering, or low resolution. Scene representations are robust enough to find matching scenes in Internet-scale photo collections even in the presence of dramatic blurring. These matching scenes provide a context-specific statistical model which can be used to insert convincing texture and object detail. Second, we are studying attribute-based representations of scenes. We use crowdsourcing to discover attributes and build large databases for the community. Attributes are a powerful intermediate representation for the next generation of big data imaging research which can have broad societal impact through applications such as robotics, security, assistance to vision-impaired, and vehicle safety. The investigators also are developing a new introductory course for Brown students to explore big data computing across scientific disciplines and are creating an online community for visual computing education to benefit students interested in photography and programming.
