The University of California-Riverside is awarded a grant to develop a new research para-digm in which both analysis-by-synthesis and synthesis-by-analysis are integrated in a closed-loop learning framework for developing robust, scalable systems for video mining in network of cameras. This paradigm allows a principled transition from limited domains to large-scale de-ployment. The collaborative project brings together three institutions (UCR, UCLA, and SUNY-SB) for aggregating and interpreting information and discovering patterns of human behavior from multiple video streams and evaluating them in realistic virtual and real-life scenarios such as video surveillance, traffic monitoring, and elderly care. The project introduces four novel elements. First, it develops methods for incremental model-ing and scaling of Bayesian nets for videos and glues together the analysis and synthesis. Sec-ond, it employs multiple strategies based on game theory and a multi-objective optimization framework for cooperative and distributed on-line control of active cameras. Third, it uses multi-ple representations in a framework of hierarchical Bayesian and Markov random fields and sta-tistical tensor models for learning long-term models of activities. Finally, it involves new models based on dynamical systems theory for seamless tracking and recognition in a video network. The project blends the theoretical and algorithmic contributions with the development of a prototype that integrates a network of video cameras with a virtual vision simulator which incorporates sophisticated artificial life models of humans. It builds increasingly sophisticated models of humans, vehicles, context, illumination, texture, shape, and motion over time. The software tools integrating the analysis and synthesis are widely disseminated.
