Detailed three-dimensional models of urban environments provide critical information for many applications, including emergency response preparation, security analysis, urban planning, and augmented-reality maps. For example, if 3D models of complete cities were publicly available with detailed labels for all semantic objects (e.g., buildings, fire hydrants, fire escapes, doors, windows, trees, etc.), then fire fighters, police forces, and other emergency response teams could use them to make plans for rescue operations, taking into account possible access points, lines of sight, and risks to the neighborhood. Or, if the 3D model contained labeled representations of stop lights, traffic signs, parking spaces, store locations, mailboxes, and ATMs, then augmented reality displays could help people navigate their daily lives.
The research goal of this project is to develop algorithms to build detailed, labeled 3D models from currently available data. Several companies (e.g., Google, Nokia, Microsoft, etc.) are currently collecting photographic imagery and LIDAR data with scanners mounted on cars driving up and down streets of cities throughout the world. This data contains a vast amount of information about our world, but in a very primitive form: pixels and points. The PI is developing algorithms to analyze this raw data to build semantically labeled 3D models: 1) new methods for discovering correspondence relationships between heterogeneous data types, focusing on LIDAR, images, and 3D polygonal models found in online repositories, 2) new ways to infer surface geometry, segmentations, and labels simultaneously based on a model learned from examples, 3) new interactive systems to allow users to visualize and guide the algorithms as they operate by incorporating user input into incrementally updated solutions, and 4) data management algorithms for multiresolution storage, compression, and retrieval of massive scanned 3D data sets.
The broader goals of the project include educational programs, industrial collaboration, free distribution of software and data sets, and outreach activities. Besides the published research results, the PI will disseminate 3D models of major cities that can be used directly in applications developed by other people. He will also distribute code, benchmark data sets, and statistical models that could benefit researchers in a variety of disciplines. This proposed work is integrated with educational programs, including interdisciplinary workshops and courses at the graduate, undergraduate, and professional levels, and diversity enhancement programs that promote opportunities for disadvantaged groups
