This project develops new methods for non-parametric prediction, filtering, and structure discovery, primarily for spatio-temporal data but also in a range of other settings with high-dimensional observations, such as networks. There are two novel aspects to the investigation's approach. First, rather than trying to predict spatio-temporal data globally, or according to a fixed pattern, it exploits the dynamics of the system to develop a novel form of local prediction which still captures long-scale structure. Second, while conventional non-parametric smoothing is based on the usual geometry of the space of predictor variables, this is supplemented smoothing together input points which have similar predictive consequences, in effect discovering a new geometry. This approach, which draws on earlier work on information theory in nonlinear dynamics, allows for accurate forecasting of the evolution of large spatio-temporal systems in a computationally efficient manner. It also allows for the automatic discovery of complex higher-level structures in such data.
Scientific data increasingly comes as complex measurements spread over space and time. Scientists need ways to forecast how such systems will evolve, and to automatically separate important (but perhaps subtle) patterns from inconsequential "background" of the system, since the structures are often crucial to understanding the dynamics. This project tackles both of these challenging statistical problems together. It combines idea from information theory and nonlinear physics with modern tools of flexible statistical modeling to discover the intrinsic dynamics of the system from the data itself, and uses these structures for both prediction and filtering. Areas of potential application include neuroscience, fluid dynamics, and ecology, where it would help forecast the behavior of complex systems, and help to find the organized structures which are keys to controlling that behavior.
