"This award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5)."
The ability to reason about the complexity of living organisms in diverse environments is one of the hallmarks of intelligence. In this project the PI and her interdisciplinary team of investigators will design computer vision algorithms for intelligent tracking of large groups of living individuals in three-dimensional space. She will develop specific systems for tracking groups of microorganisms, bats, birds, and humans. And she will formulate machine learning methods for analyzing group behavior, specifically the conditions for formation and dispersal of groups, and the interactions of individuals within a group. An important innovative aspect of this research is the systematic and comprehensive approach to reasoning about the motion of large groups of living organisms observed in video data, independently of whether they happen to be humans, animals, or cells. Previous efforts in this area have typically focused on studying the behavior of a single type of organism, and on testing theories of behavior based predominately on simulations, without the appropriate analytical tools to automatically explore and quantify the vast number of visual data sets. This project, on the other hand, will base research findings on the analysis of thousands of trajectories of individual group members moving in 3D space. To this end, the PI and her team will collect video data in the field and in public spaces to ensure optimal data capture conditions. They will use these data to develop robust solutions for the problem of matching hundreds of individual bats, birds, or people from frame to frame. They will generate stereoscopic reconstructions of movement trajectories based on multiple calibrated cameras, and use machine learning to model group behavior and mine the trajectory data. Finally, they will compare the findings of their reasoning system against current theories about the formation of groups and the interactions of individuals within a group. A similar, systematic research strategy will be employed to address understanding of the behavior of single cells. The team will design microscope imaging protocols, develop solutions for the segmentation and tracking of individual cells, and use statistical learning techniques to discover patterns and correlations in the behavior of the cells on physiologically relevant substrates.
Broader Impacts: Understanding the processes by which groups of animals and microorganisms behave is crucial to the effective conservation of populations and ecosystems and the management of cellular environments. Project outcomes will advance knowledge across the fields of computer vision, artificial intelligence, behavioral ecology, and biological engineering, and will provide new tools for answering urgent economic and ethical questions, for example about the mortality of birds and bats in wind energy facilities.
