This research addresses interactivity and scalability in automatically analyzing large collections of video and multiple video streams processed continuously. This work is developing mechanisms to enable real-time interactive video search for user defined concepts using intelligent, active processing clusters and methods for performing high-accuracy semantic video analysis from large amounts of weakly-labeled video over distributed computing resources. The methods leverage modern cluster file systems where data is stored on the local disks of the compute servers, and the location of data is made available to the runtime system to allow co-location of compution and storage.
The specific research objectives are to allow co-location of compute and storage through a runtime for parallel stream processing that parallelizes data processing and machine learning tasks across a cluster of multi-core compute nodes. The project also extends distributed versions of graphic model algorithms to speed computation of both the basic low-level signal processing steps and for the semantic analysis based on weakly labeled video data as currently available on the web. The main outcome is to demonstrate vastly accelerated, complete processing of parallel live video streams into a retrieval database with immediate search capabilities and accessing cluster resources during interactive search. The goal of this work is to develop principles for interactive applications driven by real-time processing of high-rate streaming data. The processing architecture and modules developed in this work will enable computer vision and multimedia developers to efficiently apply and test their own methods within this framework.
This research addressed interactivity and scalability in automatically analyzing large collections of video and multiple video streams processed continuously. This work developed mechanisms to enable real-time interactive video search for user defined concepts using active processing clusters and methods for performing high-accuracy semantic video analysis from large amounts of weakly-labeled video over distributed computing resources. The methods leverage modern cluster file systems where data is stored on local disks of the compute servers, and the location of data is made available to the runtime system to allow co-location of compute and storage. One specific research objective was to allow co-location of compute and storage through a runtime for parallel stream processing that parallelizes data processing and machine learning tasks across a cluster of multi-core compute nodes. Another achievement was to extend distributed versions of common analytic algorithms to speed computation of both the basic low-level signal processing steps and for the semantic analysis based on weakly labeled video data as currently available on the web. A main outcome has been to demonstrate vastly accelerated, complete processing of parallel live video streams into a retrieval database with immediate search capabilities and accessing cluster resources during interactive search for healthcare observations. The main impact of this work has been to develop principles for applications driven by real-time processing needs of high-rate streaming data. The processing architecture and modules developed in this work enable computer vision and multimedia developers to efficiently apply and test their own methods within this framework. In addition to a number of peer-reviewed publications in major journals and conferences, the project participant were involved in organizing four workshops around key themes of the research, namely, the 1st workshop on multimedia corpus in 2009, the workshop on analysis and evaluation of large-scale multimedia collections, the worshop on the future of multimedia analysis and mining in 2010 and the workshop on the future of mutlimedia analysis in 2012 and mining and the workshop on Multimedia Information Indexing and Retrieval for Healthcare in 2013.
