Recently, reconstructing full and detailed 3D models for large-scale urban environments has become the crucial technology for many applications and offers a natural platform for different services. Although conventional structure from motion (SFM) techniques have been engineered to their maturity, they normally do not utilize rich global structures of urban scenes, including regularity, symmetry, and self-symmetry; and the very repetitive shapes and textures ubiquitous in urban scenes make detecting and matching features extremely challenging for the conventional techniques.
This project develops a novel approach for inferring highly accurate 3D geometry from an individual or multiple 2D images of an urban scene. It takes full advantage of the rich global symmetry and regularity in urban scenes by leveraging powerful computational tools from modern high-dimensional convex optimization. The developed method can accurately recover the regular 3D geometry and 2D texture of the scene directly from the raw image pixels/regions without relying on extracting any intermediate local features. The research includes developing a set of useful tools and a full system that can significantly improve the efficiency and scalability of 3D modeling of large urban scenes and give significantly more compact representation of the 3D geometry and 2D appearance, enabling online real-time rendering and visualization.
Research results from this project can be easily integrated into and significantly improve the current computer vision course on 3D reconstruction. The associated technologies can be useful for a very wide range of commercial applications such as online or mobile visual search, visual guidance, navigation, or surveillance, virtual tourism, and augmented reality etc.
