The development of effective algorithms for analyzing and characterizing the behavior of groups of individuals requires that two main challenges be addressed. The first challenge is to represent the group behavior as the interaction of the behavior of individuals. This interaction can be complex, with coordinated behavior being: simultaneous, sequential, or temporally separated; driven primarily by a prearranged script or through signaling cues (e.g., visual, auditory, and electronic); and spatially localized or distributed. Lastly, the breadth of possible coordinated behavior requires strategies to create robust representations. The PIs will develop a sensor-independent symbolic description of human behavior that is readily derivable from sensor data yet can be algorithmically manipulated. This effort will develop a comprehensive framework for inferring behavior of individuals and groups of individuals using hierarchical interpretation of probabilistic context free grammars, a formalism originally developed for speech understanding. This framework will take low-level sensor measurements such as location, motion and interaction with the environment to hierarchically construct models of what is happening in the physical world, exploiting analogs between behavior understanding and speech understanding. The second challenge is to develop strategies to manage computational complexity, both the selection complexity of identifying an unknown number of conspirators out of a large background population, and the association complexity of managing the uncertainty of associating the newly observed individual behaviors with established behavioral trajectories, given imperfect tracking of individuals. Multiple strategies will be developed and applied to address this challenge. The selection complexity will be addressed by clustering the population into sub-populations based on statistical metrics, behavior classification, spatial distance, and line-of-sight constraints. The association complexity will be addressed by exploiting unique features of conspiratorial behaviors to relax the requirements on perfect data association in tracking. Successful development of these capabilities will have substantial benefits to the automated monitoring, analysis and classification of group behaviors in many potential application domains. These application domains include crowd assessment and control, riot prevention/response, prison safety, VIP protection, and building/installation defense. In addition, the algorithms and strategies developed will apply to group behaviors that are not limited to physical movements, but can extend to other forms of behavior, such as electronic communication, financial transactions, and purchasing behavior.
