In the past few years, deep learning has become the dominant approach for computer vision, image processing, speech recognition, and many other applications in machine learning and data science. This success is a synergy of several ingredients: (1) neural networks (NNs) as a flexible framework for representing high-dimensional functions and maps, (2) simple algorithms such as back-propagation and stochastic gradient descent for tuning the model parameters, (3) highly effective general software packages such as Tensorflow and Pytorch, and (4) unprecedented computing power. Despite the successes however, several key challenges remain: (1) the NN architectural design is still an art and it lacks basic mathematical principles in many cases, and (2) the NN training often requires an enormous amount of data, which is not available in many applications. Many computational problems in physical sciences also face the same challenges as those in data science: high-dimensionality, complicated or unspecified models, and large computational cost. Some well-known examples are many-body quantum systems, deterministic and stochastic control, molecular dynamics, uncertainty quantification, and inverse problems. It is natural to leverage the recent developments of NNs in the study of these problems. This project focuses on inverse problems, that is, recovering unknown problem parameters from observation data. It is a field of enormous importance, with applications in physics, chemistry, medicine, earth sciences, and defense. However, many inverse problems are known to be computationally challenging. This project will use deep learning as a means for solving these inverse problems more efficiently. The PI plans to develop a new applied linear algebra course with a machine learning focus and to organize workshops at the interface of deep learning and physical sciences. The project will train graduate and undergraduate students through the research and support one graduate student per year.
As a tool for representing high-dimensional maps, neural nets (NN) offer a flexible way for representing the full inverse maps and learn the data distribution prior via training. The rich mathematical and physical theories behind inverse problems provide theoretical guidance for designing compact yet effective NN architectures and hence it is possible to avoid the need for an enormous amount of data. In the theoretical part, this project plans to identify the commonly-used mathematical operators in many inverse problems and design novel NN modules for these operators. Two such examples are the pseudodifferential operators and the Fourier integral operators. Using analytical results from partial differential equations and numerical linear algebra, both types of operators can be represented compactly and accurately as NN modules. In the application part, this project considers five inverse problems: electric impedance tomography, optical tomography, inverse acoustic/electromagnetic scattering, seismic imaging, and travel-time tomography. The NN for the inverse map is then assembled using the modules from the theory part, along with existing primitives such as convolutional NN. The whole NN is then trained end-to-end with the training data. The research results such as publications and software will be made available in public domain. The project will also lead to broader impacts in several areas. First, this project advocates for a more principled approach for NN architecture design based on mathematical theories, sparsity considerations, and invariance/equivariance principles. Second, the project covers inverse problems from five different topics. By working with scientists and engineers, the PI plans to apply the technologies developed to practical applications.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
