This project aims to advance stereo vision and optical flow algorithms to work in challenging real-world conditions. It contributes novel algorithmic approaches, new high-resolution datasets with ground truth, and an update to the Middlebury benchmarks.
The algorithmic advances include fast matching algorithms that employ radiometric and geometric self-calibration, including smart data terms that establish noise and color models during the matching process, and novel layer-based surface reconstruction algorithms with explicit reasoning about half-occluded regions, reflections, and transparency. The new datasets reflect current challenges, including high-resolution images of real-world scenes with complex occlusions, specular surfaces and reflections under different illuminations and taken with different cameras. Undergraduate students are actively involved in all components of this research.
The project has strong potential impact along several fronts. The new datasets and benchmarks challenge the community and serve as catalysts for new research. The algorithmic advances allow harnessing the explosion of images available online, and enable real-world applications such as automated driving, geolocation, and automatic 3D reconstruction of whole cities. Finally, the project exposes undergraduates at a liberal-arts college in rural Vermont to the world of research, experimentation, and discovery.
