This project studies the fundamental mathematics of how to describe the motions visible in a video recording. The developed techniques allow accurately delineating the boundaries between image regions that move differently from each other. The resulting description of motion can be used as input for recognizing activities in video. Applications include surveillance, traffic monitoring, video retrieval, robot navigation, assistance to human vehicle drivers, medical diagnosis of movement pathology, assessment of performance in sports or other activities, sign language recognition, and automatic video annotation.
Current approaches define visual motion as a point-to-point mapping across video frames. However, image data in poorly textured areas constrain point motion weakly if at all. Since these areas are pervasive, computing point-to-point motion requires strong and often arbitrary assumptions about the scene. This project redefines image motion as a curve-to-curve mapping. The curves in question are iso-contours, that is, the curves in each video frame along which image brightness is constant. Techniques from computational topology are used and extended to describe how iso-contours in one frame connect to those in the next, forming surfaces in spacetime. The concept of persistence from computational topology, together with a new notion of feature longevity, allow separating ephemeral changes caused by image noise or lighting artifacts from features that reoccur consistently over time. The research can provide a global, topological, stable description of image motion. The research team evaluates the techniques on both existing video and on sequences newly recorded with specialized cameras to isolate different technical challenges in turn. Other researchers can use these sequences for further experimentation when they are ready to be published.
