This project explores three new related approaches to making the indexing and retrieval of videos (1) more efficient by developing rapid feature selection, (2) more meaningful by devising measurably useful indexing ontologies, and (3) more humanly navigable by demonstrating integrated multimedia browsers, even when the videos are unedited.
The first approach consists of heuristic adaptations of machine learning algorithms to the overly redundant data that is video. The second approach consists of using statistical tools to examine the descriptive tags that can be affixed to video segments, in order to measure and refine their quality. The third approach consists of exploiting the first two approaches to discover and display the weak structure that is latent even in the unedited videos of student presentations. The experimental research will be refined by continuing user studies, involving students from middle school, high school, college, and post-graduates.
The results of this project will provide advances at the multiple intersections of computer vision, machine learning, data management, information retrieval, ontology design, user interface technology, and user studies, with possible applications in the wider areas of sensory retrieval in general.
Broader impacts: We expect that the browser will enhance the effectiveness of undergraduate education by allowing accurate and rapid review of both instructor and student recorded presentations. We also expect that the underlying novel technologies will permit real-time analysis and access of more standard videos. The project Web site (www.cs.columbia.edu/~jrk/unstructured) will be used to disseminate resulting publications, open-source code, and instructions on how to obtain annotated video data sets.
