Finding highly correlated latent factors in multimodal signals and data: Scalable algorithms and applications in sensing, imaging, and language processing
Abstract: Multimodal signals and data arise naturally in many walks of science and engineering, and our digital society presents ever-increasing opportunities to collect and extract useful information from such data. For example, brain magnetic resonance imaging and electro-encephalography are two modes of sensing brain activity that can offer different "views" of the same set of patients (entities). Co-occurrence frequencies of a given set of words in different languages is another example. Crime, poverty, welfare, income, tax, school, unemployment, and other types of social data offer different views of a given set of municipalities. Integrating multiple views to extract meaningful common information is of great interest, and finds a vast amount of timely applications -- in brain imaging, machine translation, landscape change detection in remote sensing, and social science research, to name a few. However, existing multiview analytics tools -- notably (generalized) canonical correlation analysis [(G)CCA] -- are struggling to keep pace with the size of today's datasets, and the problem is only getting worse. Furthermore, the complex structure and dynamic nature of some of the underlying phenomena are not accounted for in classical GCCA. This project will provide much needed scalable and flexible computational tools for GCCA-based multimodal sensing and analytics, thereby benefiting a large variety of scientific and engineering applications. It will produce a framework allowing for plug-and-play incorporation of application-specific prior information, and distributed implementation. Beyond linear and batch GCCA, nonlinear GCCA and streaming GCCA will be considered. These are appealing and timely for many applications, but associated computational tools are sorely missing.
In terms of theory and methods, many key aspects of GCCA (such as convergence properties, distributed implementation, and streaming variants) are still poorly understood. The research will provide a set of high-performance computational tools that are backed by advanced optimization theory and rigorous convergence guarantees. The research will evolve along the following synergistic thrusts: 1) scalable and stochastic GCCA algorithms; 2) distributed, streaming and nonlinear GCCA algorithms; and 3) validation, using a series of timely and important applications in remote sensing, brain imaging, natural language processing, and sensor array processing. Devising scalable, flexible, streaming, and nonlinear GCCA algorithms is very well-motivated for modern sensing and analytics problems which involve rapidly increasing amounts of data with unknown underlying dynamics. Using GCCA for large-scale dynamic and complex data also poses very challenging and exciting modeling and optimization problems.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
