This project deals with theory and efficient algorithms for statistical decision problems that are radically different from those that have been studied to date in two key aspects: First, the decision-maker may choose among a large class of observation channels (features) of varying complexity and quality; and second, the total cost of computational resources that can be used prior to arriving at a decision is limited. Computer vision is a paradigmatic source of such feature-rich decision problems, requiring the use of multiple heterogeneous feature types, integration of diverse sources of contextual information, and possibly even human interaction.
This project entails the development of a rigorous mathematical framework for feature-rich decision problems in accordance with three specific aims: (1) structural characterization of features as stochastic belief-refining filters; (2) universal cost-sensitive criteria for numerical comparison of features in terms of expected information gains; and (3) optimal value-of-information criteria for sequential feature selection that take into account both feature extraction costs and terminal decision losses. As corollaries, this research investigates connections to asymptotic information-theoretic characterizations of optimal feature selection rules and decisions. The fourth specific aim of the project is the development of practical algorithms for two challenging computer vision problems: active visual search and fine-grained categorization. This component of the project leverages theoretical aims (1) and (2) to develop practical cost- and loss-sensitive feature compression techniques. Theoretical aim (3) targets algorithms that function as autonomous decision-making agents. Faced with an inference task on an image, they apply cost-sensitive non-myopic value- of-information criteria to decide at each time step whether to extract a new feature from the image or to stop and declare an answer.
