Motion is an important component in temporal datasets representing a variety of sensor devices. This project investigates the design of scalable motion-content-based indexing/retrieval mechanisms and an activity recognition system for applications that employ large motion data archives. A unified mathematical framework is developed for representing motion features that allows integration of indexing/retrieval formulation as well as semantic-based intelligent recognition systems. Motion trajectories are segmented into sub-trajectories using computationally efficient techniques that exploit curvature information and cope with occlusions and missing data. Representation of sub-trajectories is based on principle component analysis (PCA) of the motion data which allows compact low-dimensional representation as well as real-time query processing. Extension of this representation to multiple motion trajectory data is based on three-dimensional tensor singular value decomposition (SVD). Innovative use of these analytically-motivated feature spaces is relied upon for developing robust indexing and retrieval systems and scalable activity recognition systems based on Hidden Markov Models. The techniques developed are prototyped and the performance of the system is evaluated using several video archives. The project is a giant step forward towards unifying query-by-example-based indexing and retrieval systems and high-level semantic query-based activity recognition systems. This project will significantly enhance the current state of the art in content-based indexing and retrieval and activity recognition systems for applications that employ temporal datasets. It will facilitate the development of diverse motion-based applications for entertainment and security applications. The project web site (http://multimedia.ece.uic.edu/motionsearch) provides access to resulting research papers and implementation code.
