The effort focuses on developing recognition algorithms that use a large number of multimedia sensors that are deployed as needed in an ad hoc fashion. The goal is to design algorithms that connect the various uncoordinated views and deduce the global state. Rather than requiring real time scene analysis, the proposed system stores the scenes in high fidelity for specific durations in in-situ storage bricks. Retrospective algorithms search and leverage the various views in order to detect complex states that are missed by analyzing a single stream. Storing the stream also allows the system to analyze complex behaviors that require behavior analysis over extended periods of time. A self managing infrastructure for profuse media capture and storage will be developed including efficient mechanisms for the sensor components to search and locate other components using an unstructured peer-to-peer overlay. Operating systems mechanisms that allow these sensor components to balance their local resource consumption with global end-to-end requirements will be designed. Pattern recognition tasks such as semantic tagging of actor trajectories, multi-camera moving object tracking as well as mechanisms for integrate multimedia data sets to extract patterns of interest are also planned.
The proposed research will enable recognition tasks that are not possible in traditional system that require real time processing. The focus on sensors built with commodity components offer the potential for immediate impact to the user community. These systems promise to have significant economic, environmental, and societal impacts. The research prototypes developed will be disseminated freely. The educational aspect of this proposal will focus on integration of the research results into an experimental systems curriculum with graduate training and undergraduate summer research mentorships, especially for women and underrepresented students.
