The literature on the formulation, analysis and applications of convex optimization is ever expanding due to its broad applications in signal processing, machine learning, statistics, and other fields of data science. In theory, many convex problems have a well-understood structure, and hence state-of-the-art first order and interior-point methods can obtain high accurate solutions. In practice, however, modern applications present a host of increasingly larger-scale and nonsmooth optimization problems that can render these methods impractical. Fortunately, recent advances in convex optimization offer a surprising new angle to fundamentally re-examine the theory and practice of large-scale convex optimization models in a unified fashion. Successful development of the PI's ideas will have several broad impacts in data analysis and computational science. While existing state-of-the-art approaches focus on certain classes of convex problems, the PI strongly believes that the approach proposed in this project can be expand to cover a wide range of unexploited convex optimization applications. The obtained theory and methods can be specified and customized to solve various problems in different fields, including massive data analysis, machine learning, high-resolution imaging science, operations research, networks, and control. Successful real-world applications and software development can create a major impact to practicians in academic and industry. The PI's broad collaborations are expected to make a significant progress in the application of convex optimization techniques. Several research topics in this project will be integrated into graduate training programs through special topic courses and PhD research directions, whereas undergraduate training activities can also benefit from this research via internship training and interdisciplinary collaborations.
This project focuses on exploiting and generalizing a prominent concept so-called self-concordance to develop new efficient convex optimization techniques to attack two classes of large-scale convex optimization problems, and will be integrated into three work packages (WPs). WP1. Composite self-concordant convex optimization: While existing convex optimization methods essentially rely on the Lipschitz gradient assumption which unfortunately excludes many important applications such as Poisson and graphical learning models, the PI instead focuses on the self-concordance structure and its generalizations. Such a concept is key to the theory of interior-point methods, but has remained unexploited in composite minimization. Grounded in this structure, the PI will develop novel and provable convex optimization algorithms for solving several subclasses of large-scale composite convex problems. He also plans to generalize this self-concordant notion to other subclasses of problems such as logistic loss functions and entropy models to cover a broader range of applications. WP2. Constrained convex optimization involving self-concordant barriers: Various constrained convex applications are integrated with a self-concordant barrier structure, while other convex constraints often have a "simple" structure. Existing general-purpose convex algorithms solve these problems by mainly employing either a standard interior-point method or an augmented Lagrangian framework. The PI alternatively concentrates on exploiting special structures of these problems using the theory of self-concordant barriers and combining them with both the interior-point idea and the proximal framework to develop new and scalable algorithms equipped with a rigorous convergence guarantee, while offering a parallel and distributed implementation. WP3. Implementation and applications: This WP aims at investigating the implementation aspects of the proposed algorithms and upgrading the PI's SCOPT solver. The methods developed in this project will be validated through three concrete real-world applications: image processing involving Poisson models, graphical learning problems, and large-scale max-cut and graph clustering problems.
