This proposal targets the problem of non-rigid structure from motion (NRSFM). This is a badly ill posed problem, as none of the usual multiple view geometric constraints hold, and additional constraints are necessary. The PI presents a research agenda to solve this problem. The first approach is based on a temporal relaxation of rigidity, and finding the solution that minimizes the total variation in scene structure over time. The second approach instead regards a video of a non-rigid object as a sequence of images of rigid shapes under varying pose, leading to a joint clustering and SFM problem.
Intellectual merit. For the SFM enterprise to realize its full utility, it must account for important non-rigid cases. Apart from being an important and challenging open problem, the estimation of NRSFM has characteristics that place it at the cutting edge of research in computer vision: (1) it requires a deep understanding of statistical modeling of shape deformation, and (2) the underlying math is at the intersection of variational methods, graph theory, optimization, differential geometry, and numerical linear algebra. Progress in this area will involve technical advances that will benefit a wide range of problems in computer vision.
Broad impact. Scientific: Practical applications of NRSFM include nonrigid object recognition, motion capture of humans and animals, gesture recognition (e.g., for American Sign Language), and medical imaging (e.g., contraction of a heart ventricle). Educational: The PI presents plans to provide the supporting educational infrastructure at the undergraduate and graduate levels in coordination with UCSD's NSF IGERT training program in Vision & Learning. This consists of new courses covering image processing, computer vision, and machine learning, and an outreach effort for underrepresented students through the UCSD Summer Training Academy for Research in the Sciences (STARS) program.
