This project develops a fragment-based intermediate-level representation for images based on shape and appearance, with shape playing the primary role. The representation can be populated in a bottom-up, category-independent fashion, which at the same time can be efficiently accessible by top-level, category dependent processes. The inherent ambiguity in generating object part hypotheses is resolved by combinatorially forming alternative image fragments by standard as well as novel perceptual operations, taking into account both shape and appearance, and both region-based and boundary-based cues. The exponential growth of the number of fragments is managed under a best-first graph representation that avoids duplication, leading to a sufficient number of diagnostic recognizable object parts among a vast pool of fragments. The project also explores an embedding of these fragments in a metric similarity space via proximity graphs and a geometric index structures for efficient nearest neighbor search. The final outcome is a representation space and an index for scalable, logarithmic object category recognition. A key aspect of this work is that categories themselves are also represented in a hierarchical similarity space, and this computationally implements ideas akin to Rosch?s basic level categorization.
The broader impacts of this activity spans a vast number of applications: any application which benefits from scalable object recognition such as indexing into databases, e.g., searching in a database of trademarks, engineering drawings and computer generated graphics, content-based web search, aerial tracking and recognition of vehicles, automated animal behavior analysis, and many others.
