This award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5).
The objective of this research is to develop bio-inspired algorithms for recognizing human movements and movement styles in videos of human activities. The approach is based on a new representation of static three-dimensional (3D) shape structure in the ventral visual pathway consisting of configurations of 3D structural fragments. This project uses neural experiments to validate an analogous four-dimensional (4D) representation of moving 3D shapes based on 4D (space and time) structure-in-motion (SiM) fragments. These SiM fragment models are used to develop algorithms for automatically extracting SiM fragments from videos of human activities. Hybrid system identification and clustering techniques are used to learn a dictionary of human movements used for recognition. This dictionary, in turn, influences the design of the neural experiments. The algorithms are evaluated using a real-time tele-immersion system.
The intellectual merit of this project is to substantially reduce the gap between human and machine perception of human movements through the interaction between computational and experimental analyses from neuroscience, computer vision, machine learning, and dynamical systems. This project also has the potential to generate advances in machine learning and dynamical systems, by extending classification, clustering and system identification methods to time-series data generated by collections of hybrid dynamical models.
Potential broader impacts include applications in surveillance, security, assisted home living, infant care, tele-immersion, and athlete motion analysis. The project also sponsors a competition where small robots are constructed by middle and high-school students from Baltimore, and the EL Alliance program for retaining underrepresented minorities at graduate schools.
