Blind source separation (BSS) refers to the task of identifying sources from their linear mixtures. Traditional approaches to BSS have been limited to static mixtures. Furthermore, such approaches typically rely upon hard-to-exploit and non-robust assumptions on source-statistics. In contrast, the proposed research addresses the general problem of separating dynamically-mixed signals by simultaneously identifying both the dynamics as well as the input sources. The basic tool in the formulation of relevant ill-posed system identification problems is the notion of sparsity which is used as a regularization term to limit the choices of input/process dynamics in a natural way. The proposed research stands to benefit from a rather powerful theory on computationally-tractable sparsity-inducing optimization, based on ℓ1-functionals, which has taken shape in recent years.
The proposed plan begins with an analysis of a general dynamic-mixtures-model, exploring sparsity as a regularizing term. Motivation for such models stems from system identification, distributed sensing, as well as problems in spectral analysis, subspace identification, and antenna arrays. The proposal continues on with an outline of specialized formalisms intent on capturing, in a similar framework, problems of delay/coherence analysis as well as of system identification in a non-stationary/nonlinear-mixing setting. To this end, it is proposed that the notion of joint sparsity?a form of dependent-component-analysis, is a suitable tool for identifying commonalities between sources, harmonics, etc., while seeking tell-tale signs of the presence of time-delays and of nonlinear mixing. The proposal covers in some detail the case of autoregressive dynamics which leads to a convex optimization problem. Tradeoffs between noise, model order, and stability are raised and integrated into the proposed research plans. Connections between BSS and image segmentation techniques?a form of geometric BSS, are highlighted in a way which suggests another conceptual angle for the proposed research. Finally, the issue of dictionary design is being discussed, i.e., how to obtain a suitable ?over-complete? basis for source signals and possibly system dynamics as well, based on prior information and on available data, in a way that will ensure a degree of robustness and computability while promoting sparsity.
Intellectual Merit: Practical as well as theoretical questions will be investigated with regard to the rather ubiquitous identification problem for system dynamics and signal transmission paths, in the presence of unknown disturbances and inputs. The formalism is cast in the context of blind source separation, and the basic new tool is the concept of sparsity with respect to suitably chosen collection of signals as a selection rule for modeling. The approach stands to benefit from the theory of sparse representations/compressive sensing which has come to fruition in recent years. Problems of delay estimation, coherence analysis, non-linear and non-stationary modeling are presented with a new angle?seeking relevant information in a jointly-sparse representation of measured time-series. A potentially transformative broad spectrum of tools may result from the new ways of analysis and system identification proposed herein.
Broader Impact: The research may impact very different fields such as Physics?in calibrating and filtering measurements, Image analysis?in MRI/medical imaging, System identification, Acoustics and the control of jitter, Communications?blind deconvolution in noisy and resonant channels, Radar processing, and others.
Dynamic blind source separation suggests the ability to distinguish signals in a noisy dynamic environment. The signals are dynamically altered though partially unknown interactions and the task is to detect and identify their presence and spectral content. Prime examples include the case of sinusoidal signals (pure tones) in colored noise (broadband signal) on one hand, all the way to signals with a less structured spectral content as in patterns of physical significance (weather, deposition of sediments) which are partially observed and need to be reconciled with partially known dynamical processes (advection-diffusion). An overarching theme that covers the specifics of the research being carried out is that of signal analysis and system identification. Blind source separation suggests a combination of the two being present in the problem at hand. The salient feature of the needed task to identify signals and dynamics at the same time requires that prior information in some form is available, and that a simple model amongst available alternatives is sought to explain observed data. During the duration of the research that has been carried out a variety of complementing issues has been considered and theory and new techniques have been developed. In the early stages of the research new sparsity inducing techniques were employed in conjunction with rank surrogate functionals so as to cast the problem at hand as an optimization one. In simple terms, among the class of possible models for both dynamics and signals, and optimization criterion is set to discern and choose the simplest option (Occam’s razor principle). Signals are to be selected from a known or, approximately known dictionary of possibilities and dynamics are to be selected so as to rely on a small number of parameters. The former is achieved by the so-called sparsity induced functionals (i.e., few non-zero selection entries) and the latter through rank-surrogate functionals (i.e., convenient functionals that penalize large dynamical complexity). After initial success in certain range of somewhat idealized scenarios, it became apparent that a deeper study of metrics that quantify uncertainty in various ways became necessary. To this, a range of alternative situations were considered and metrics as well as techniques for solving inverse problems were developed. In particular, the research has led to new insights on using the so-called optimal-mass-transport metric (Wasserstein metric) in model/signal identification. The important feature of the metric which was suitable adapted in the course of the research is that it "respects" the physical significance of the energy/amplitude content of signals. Thus, signals with similar support (in time or space or frequency) ought to be near as intuition suggests, in contrast to quadratic metrics that are typically used. A very important advance has been the generalization of these ideas to multivariable-transport (so as to compare matrix-valued distributions) where both energy content as well as directionality are important. A range of topics that include studying properties of power spectra, completion of partially known statistics to abide by suitable functionals, introduction of new metrics (transport) for assessing resolution/uncertainty have been accomplished. The first figure is from work on "Coping with model error in variational data assimilation using optimal mass transport" ( Ning, L., Carli, F. P., Ebtehaj, A. M., Foufoula?Georgiou, E., & Georgiou, T. T. (2014), Water Resources Research, 50(7), 5817-5830), a work that integrated the concept of a transportation metric with the need to quantify uncertainty and correct for possible biases in advection-diffusion models for data assimilation in e.g., hydrologic and atmospheric systems. It represents a simple 1-dimensional academic example where the techniques can be checked agains "ground truth" (solid blue) and shows that a technique utilizing mass-transport metric to quantify discrepancy in the measurements to what an advection-diffusion model predicts corrects the model parameters almost perfectly in controlled numerical experiments; the figure compares the outcome that is achieved by using transport metrics (red dash-dot) as opposed to the more standard quadratic ones (dash green). The second figure compares the performance of matrix-valued transport metrics to discern the movement of two sources of identical frequencies as they cross paths and is discussed in the work "On Matrix-Valued Monge-Kantorovich OptimalMass Transport" ((L. Ning, T.T. Georgiou, and A. Tannenbaum, IEEE Transactions on Automatic Control, DOI: 10.1109/TAC.2014.2350171).
