This collaborative research project leverages expertise of four research teams (IIS-1111415, Massachusetts Institute of Technology; IIS-1110955, Harvard University; IIS-1111398, Washington University; and IIS-1111534, Cornell University). Understanding time-varying processes and phenomena is fundamental to science and engineering. Due to tremendous progress in digital photography, images and videos (including images from webcams, time- lapse photography captured by scientists, surveillance videos, and Internet photo collections) are becoming an important source of information about our dynamic world. However, techniques for automated understanding and visualization of time-varying processes from images or videos are scarce and underdeveloped, requiring fundamental new models and algorithms for representing changes over time. This research involves creating systems that enable modeling, analysis, and visualization of time-varying processes based on image data. These models and algorithms will form the basis for a new set of tools that can help answer important questions about how our environment is changing, how our cities are evolving, and what significant events are happening around the world.
Analyzing images over time poses fundamental new technical challenges. This project focuses on developing and demonstrating end-to-end systems consisting of (1) novel representations necessary to model time-varying image datasets; (2) algorithms for estimating long-range temporal correspondence in image datasets; (3) algorithms for decomposing image datasets into intuitive primitives such as shading, illumination, reflectance, and motion; (4) analysis tools for deriving higher level information from the decomposed representations (e.g., trends, repeated patterns, and unusual events); and (5) tools for visualization of the high-level information and methods for re-synthesis of image data.
This work has the potential to have significant impact in a broad range of areas where images are generated over time, e.g., in ecology, astronomy, urban planning, health, and many others. The results of this research will be broadly disseminated by making source code and datasets publicly available via the project web site (https://groups.csail.mit.edu/vision/image_time/) and offering tutorials and organizing workshops at significant conferences. The project provides educational opportunities and offers hands-on collaborative research experience to students at both the undergraduate and graduate levels and the four institutions.
