This proposal addresses three sets of problems in the intersection of image processing, computer graphics and computer vision: (1) image segmentation with applications to object tracking in videos, (2) inpainting problems, both in the pixel domain and the transform domain, and (3) applications of variational PDE-based models to image processing on manifolds. The approach makes use of powerful concepts from computational mathematics, such as duality, convexification, non-smooth optimization, fast combinatorial optimization, numerical PDE techniques, harmonic analysis, and computational differential geometry. The research is focusing on key issues, such as computational efficiency, feature extraction, object tracking, global analysis, and optimization, and preserving both geometric and texture information, which are keys to further advances. The research will impact many areas ranging from entertainment through homeland security and medical imaging.
For image segmentation, typical models are generally nonconvex and admit many local optimal solutions, making them sensitive to initial guesses. Moreover, numerical algorithms can be trapped in local non-optimal minima. To obtain better segmentation algorithms, we are extending a novel convexification technique (through the deep connection between TVL1 models and level sets) developed earlier for the Chan-Vese segmentation model to other segmentation models, such as non-local segmentation models inspired by texture synthesis techniques. For inpainting, recent advances include PDE and geometry techniques, texture synthesis, and a hybrid framework for inpainting missing transform (e.g. wavelets or Fourier). This research is examining the synergy between these methods and deriving models and algorithms that combine the best features of each. For image processing on manifolds, we are using conformal mapping techniques and PDE-based image processing models to derive efficient algorithms for general surfaces. New applications include the automatic tracking of landmarks on general surfaces and the incorporation of shape information in landmark matching.
The major goals of this project is to develop innovative and effective mathematical models and associated fast computational algorithms for image processing, computer vision and computer graphics applications. We developed fast numerical methods for image processing models. We consider extensions of these image models to images not just on flat domains but on general surfaces, including brain surfaces in the context of brain mapping. Some significant results that we have obtained are: We developed a fast denoising algorithms based on a novel kind of iteration called Bregman Iteration, which is essentially an augmented Lagrangian method for the constrained optimization formulation of the image model. We developed methodologies for convexifying non-convex image segmentation models, as well as for image registration problems. Convexification makes the underlying optimization problem much easier to solve. We introduced a class of color image restoration algorithms based on the Mumford-Shah energy minimization model and nonlocal image information. We applied the above ideas to high-dimensional data clustering problems, such as those that may arise in data mining and data classification problems. We developed a Boolean logic extension of the widely used Chan-Vese segmentation model to the multiphase case (multiple classes of segments), allowing the use of different, user-specified, combination of information from the different channels to direct the segmentation. Finally, we extend the image model methodology we have developed to the case of general manifolds. We developed a mathematical framework, as well as natural extension of existing algorithms, for "flat" images to surfaces. We developed a vectorial total variation method for denoising surfaces, which is designed to preserve ridges and sharp corners in the surface. We developed a framework of using conformal and quasi-conformal maps, and the use of associated Beltrami coefficients, to both efficiently represent the map (e.g. compression) and to do analysis (e.g. extracting features, detecting abnormal shape deformation). We have applied this methodology to brain mapping problems. This project supported the training of five postdocs (4 have gone on to faculty positions, 1 to industry), and five PhD students (3 have gone on to postdoctoral positions, 2 to industry).
