This project aims to use a combination of elements from dynamic vision, dynamical systems theory, optimization and semi-algebraic geometry to develop a computationally tractable, scalable framework for automatic dynamic scene understanding from multiple, potentially incomplete and corrupted data streams. The long-term vision is to lay the foundations for synthesizing provably robust vision systems, capable of sustained successful operation in complex dynamic scenarios.
The core of the project is a unified vision, centered on the use of dynamical invariants as information encapsulators, and emphasizing robustness and computational complexity issues. In this approach, the observed data is treated as the output of an underlying model, typically a difference inclusion, which has associated certain quantities (for example order, embedding manifold, subspace spanned by its trajectories) that are invariant to coordinate transformations, initial conditions, viewpoint changes, synchronization, etc. These invariants compactly capture spatio-temporal information from video data and lead to robust, computationally efficient algorithms for automatic video scene understanding. For instance, in this context video data can be efficiently segmented by detecting changes in these dynamic invariants or clustered according to a suitable defined distance. An application domain directly impacted by this research is aware environments for public space safety, where the co-PIs have been provided access to real data and given a venue for real time testing of the algorithms.
