Over the past decade, two major trends have reshaped data science: (i) a growing tidal wave of imaging and sensing devices and (ii) the rapid adoption of break-through machine learning technologies. From autonomous vehicles to medical devices, the ability to collect and analyze large quantities of data is changing the world. The goal of this research project is to create a new mathematical and computational framework for integrating distributed and multimodal sensor data into multiple types of emerging models in data science and machine learning so as to extract more information from data. The project team includes researchers from diverse disciplines who will address problems ranging from physical science and medicine to consumer imaging and industrial inspection. The research will result in theories, algorithms, and open software that can be used to integrate information from heterogeneous sensing systems to estimate and reconstruct signals and images.
The framework for model-data integration is based on a new theory of Multi-Agent Consensus Equilibrium (MACE). MACE allows for modular integration of multi-modal physical sensor information with information derived from data science models. At the core of this approach is the computational solution of the consensus equilibrium equations. These equations balance distributed sensor information with prior knowledge provided by machine learning models. The MACE framework is a generalization of the more traditional Bayesian or regularized inverse approach, but it allows for the use of non-traditional data science models such as deep convolutional neural networks in the solution of sensing and imaging problems. This project's contributions are in four areas: Thrust 1: Foundational Theoretical Methods; Thrust 2: Robust Sensor and Data Model Integration; Thrust 3: Multimodal and Networked MACE; and Thrust 4: Automated Experimentation. The project also includes integrated educational activities, engaging both graduate and undergraduate students in this research, as well as the development of new courses on consensus equilibrium and nonlinear optical imaging.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
