This project is focused on research issues associated with producing extremely detailed and accurate 3D geometry and appearance (BRDF) models at city scale from internet collections of photos from various sources containing millions of photos of enormous diversity such as viewing range and conditions, time of day and weather conditions, to name a few. The properties of urban scenes may include low-texture surfaces, reflective and transparent materials, and repeated structures that challenge existing reconstruction algorithms. The investigators will address these challenges with the aim of reconstructing several large US and foreign cities. Historical photos and virtual models may also be incorporated. There are a number of research topics associated with the project. To register photographs and recover sparse geometry at city-scale, a new, unified, structure-from-motion (SfM) algorithm will be designed to take advantage of large, parallel computing platforms. With registered photographs and sparse 3D scene points recovered by SfM, multi-view stereo (MVS) algorithms can reconstruct detailed geometric models. Novel MVS algorithms will then exploit the structure of architectural scenes and volumetric reconstruction methods will be employed to produce annotated models of exceptional accuracy and usability. Digital models are playing an increasingly important role in social, cultural and economic endeavor and are central to next-generation mapping and visualization applications. All models and datasets will be made freely available to researchers and the general public.
Until very recently, if we wanted to make a digital 3D model of a city, we faced the daunting task of using expensive, dedicated equipment such as a laser scanner to capture the data of all the topography, buildings, trees, etc. We would have had to send out teams of technicians for days or weeks at a time to scan the city and process the raw results to create a polished 3D model. This was so costly and impractical that it was rarely, if ever, done, despite the obvious utility of having an accurate and detailed 3D model of the major cities around the world. In recent years, computer scientists have developed a way of capturing the same information by using the enormous number of free digital photographs of cities that are available online on websites such as Flickr. The result is that it is now practical to make a 3D model of a contemporary city fairly quickly and at little expense. In effect, we can crowd-source the process of collecting the raw data. This project helped to improve the software available to make such city models. It also addressed the next logical question: how did the same city look in the past? Of course, we can already use the same software we use to create the model of the city as it looks today to make a model of the way the city appeared after the advent of digital photography. But what about cities whose histories go back centuries, long before the invention of the digital camera? If those cities have rich traditions of photographic and artistic documentation, then is there some way to add those historic views to our model so that we can turn back the clock and see how the city looked in earlier periods? This was the question which my study helped to address. Collaborating with computer scientists at Cornell and the University of Washington, we took the city of Rome as our test case. First, a digital model was made of Rome today using existing software and exploiting the millions of free digital photos available online. Then we accumulated a large database of pre-digital, historic photographs (dating from around 1840 to 1950), and we attempted to add the information they preserve to the digital model. Finally, we created a second database of artists' views of Rome (dating to the period of about 1500 t0 1840), and we attempted to add their information to the digital model. We focused on one of the best documented areas of the city: the section running from the Roman Forum to the nearby Colosseum. From the Renaissance until today, this has been the part of the city of greatest interest to visitors, artists, and scholars. So the visual documentation from the pre-digital period that we could find--over 700 images altogether--was extensive. Besides this practical reason, we chose to study this part of Rome because we knew that a 3D temporal model of it would be of great interest and utility to scholars of Rome. Our results have been very promising. We have shown that historic photographs and artists' views, once digitized and geo-tagged, can be registered to a 3D model of a contemporary city, at least if there are features in common between the city in its two different time periods. These results are illustrated in the two images accompanying this report. The first image shows the digital model of the Colosseum made through this project in 2013. The second image shows the same model with an historic photograph dating to the period 1857/1865 registered automatically to the scene. Such combination of historic and contemporary data in building up a 3D digital model of a city has a great number of potential applications in fields such as Archaeology, Architecture, and Urban Studies.
