"This award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5)."
Although the world is very much three-dimensional, most of today's approaches to visual object recognition essentially reduce the problem to one of 2D pattern classification, where rectangular image patches are independently compared to stored templates to produce isolated object labels within the image. This project aims to account for the three-dimensional nature of the real world by exploring qualitative geometric reasoning in terms of 3D spatial relationships between scene components, category-level object models, and global scene understanding.
The project is organized around two major research areas. Qualitative 3D scene parsing: A central part of our effort will be to develop qualitative 3D models of the scene that describe the depicted objects and surfaces and their physical relations. Grounding objects in the scene: We integrate the geometric representation of the scene and the corresponding 3D spatial relations with the object recognition process by (1) inferring the set of likely object identities based on 3D relations among scene components; (2) predicting the most likely object locations from the scene layout; and (3) using the occlusion relations and depth ordering to predict the parts of objects that may be visible in the scene.
The project is anticipated to result in major advances in 3D scene understanding from photographs, a critical enabling technology for a wide range of applications including autonomous systems, health care, human-computer interaction, assistive technology, image retrieval, industrial and personal robotics, manufacturing, scientific image analysis, surveillance and security, and transportation.
