In this research program, the PI proposes a novel combined geometric active contour/particle filtering approach for tracking the boundaries of objects (i.e., planar shapes), when the observation is an image which may be a complicated nonlinear function of the closed curve. The advantage of using geometric active contours is that they allow topological changes (automatic merging and breaking), and hence can be used to track multiple objects. More specifically, the particle filtering framework will be applied to the space of continuous closed curves which is an infinite dimensional space. This is a particularly difficult problem since generating Monte Carlo samples from a very large dimensional (theoretically infinite) system noise distribution is computationally complex. Moreover, the number of samples required for accurate filtering increases with the dimension of the system noise. The PI will show that as long as the number of dimensions of the system noise is small, even if the total state space dimension is very large (or infinite), a particle filtering algorithm can be implemented which will allow him to develop practical robust tracking algorithms. In particular, the PI proposes to approximate curve deformation using a time- varying finite dimensional representation. He will formulate the problem as particle filtering with unknown static parameters and use a modification of a particle filter that has been shown to be asymptotically stable for tracking static parameters. The main assumption is that even though the curve may be regarded as a point of an infinite dimensional space, "most of its deformation" for a given period of time can be approximated using a small finite number of dimensions. But over time, this approximation may no longer suffice and hence one must allow the number of dimensions and the finite dimensional basis to change whenever the current approximation is unable to track with suffcient accuracy. For a number of key scenarios, this assumption seems reasonable, and allows the use of infinite dimensional observer techniques for visual tracking. Intellectual Merit: The key objective of this project is the development of new methodologies for employing visual information in a feedback loop, the underlying problem of controlled active vision. This is a challenging problem both from the intellectual and practical points of view. Indeed, controlled active vision, and in particular visual tracking requires the integration of techniques from control theory, signal processing, and computer vision. This research program points the way to finding a new class of robust and hopefully real-time visual tracking schemes making use of all of the above building blocks. Broader Impact of Research Activity: The PI believes that the proposed synergy of vision, filtering, and control described in this proposal may have a strong impact on tracking and active vision. Indeed, visual tracking provides a fundamental example of the need for controlled active vision. While tracking in the presence of a disturbance is a classical control problem, visual tracking raises new issues. Firstly, since cameras are part of the system, one must consider the nature of the disturbance from imaging sensors. Secondly, the feedback signal may require some interpretation of the image, e.g., segmentation of a target from its background, or an inference about an occluder. Finally, as visual processing becomes more complex, the issue of processing time arises. Each of these problems must be answered before target detection, and visually-mediated control can be provided for medical, commercial, or advanced weapon systems.
