Three sibling algorithms, expectation-maximization (EM), mean-field variational inference, and Gibbs sampling, are among the most popular algorithms for statistical inference. These iterative algorithms are closely related: each can be seen as a variant of the others. Despite a wide range of successful applications in both statistics and machine learning, there is little theoretical analysis explaining the effectiveness of these algorithms for high-dimensional and complex models. The research presented in this project will significantly advance the theoretical understanding of those iterative algorithms by unveiling the statistical and computational guarantees as well as potential pitfalls for statistical inference. The wide range of applications of EM, mean-field variational inference, and Gibbs sampling ensure that the progress we make towards our objectives will have a great impact in the broad scientific community which includes neuroscience and social sciences. Research results from this project will be disseminated through research articles, workshops and seminar series to researchers in other disciplines. The project will integrate research and education by teaching monograph courses and organizing workshops and seminars to help graduate students and postdocs, particularly minority, women, and domestic students and young researchers, work on this topic. In addition, the PI will work closely with the Yale Child Study Center and the Yale Institute for Network Science to explore appropriate and rigorous algorithms for neuroscience, autism spectrum disorder, social sciences, and data science education.
The PI studies these iterative algorithms by addressing the following questions: 1) what is the sharp (nearly necessary and sufficient) initialization condition for the algorithm to achieve global convergence to optimal statistical accuracy? 2) how fast does the algorithm converge? 3) what are sharp separation conditions or signal strengths to guarantee global convergence? 4) what are the estimation and clustering error rates and how do they compare to the optimal statistical accuracy? There are three stages to developing a comprehensive theory for analyzing iterative algorithms: 1) studying statistical and computational guarantees of EM for Gaussian mixtures for both global parameter estimation and latent cluster recovery, 2) extending EM to mean-field variational inference and Gibbs sampling, and considering a unified analysis for a class of iterative algorithms, 3) extending Gaussian mixtures and Stochastic Block Models to a unified framework of latent variable models.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
