This project addresses three main challenges in large scale 3D modeling from photocollections: 1) efficient and complete data linkage, 2) Inference and modeling of scene and user dynamics, and 3) development of data adaptive algorithms. These challenges are tackled within the framework of data association and exploitation. The benefits of such an approach are two fold. First, through enhanced data association, the approach increases the scope of 3D models due to more complete data linkage. Second, through the development of algorithms that are not only robust against (and mitigate) input data variability, but also explicitly designed to exploit this diversity and data richness, the approach increases fidelity of 3D models. The specific data association tasks of this project include: location recognition, view planning for 3D reconstruction, online learning for feature matching, and modeling under scene symmetries. The specific data exploitation tasks of this project include: model update and archiving, native resolution modeling, high resolution dynamic texture estimation, and exploring and leveraging user behavior.
This work enables the broad deployment of applications where fully automated scene modeling expands from determining structure properties to encompass the modeling of observable behavioral patterns both in the scene and in the user controlled image capture process. The developed technologies have a wide range of applications, from virtual tourism, to cultural heritage preservation, to disaster response.
