This research explores the ability of data-derived modeling to extract, without a priori assumptions, the inherent features of a system from its time series data. Although the essential elements of this approach to modeling were developed for geospace phenomena, it is applicable to many natural as well as social phenomena. With the increasing importance of Big Data the fundamental nature of data-derived techniques provide a new approach to the modeling of systems from their time series data. Often such systems are not readily modeled using first principles and data-derived modeling can be the only viable approach. As in the case of geospace the dynamical models can lead to data-enabled forecasting tools. Another aspect of the research is the ability to quantify the variability of the system using a new fluctuation analysis, which yields improved fluctuation exponents such as the well-known Hurst index. The modeling of dynamical behavior, prediction, forecasting and characterization of is an integral part of the emerging data-enabled science.
The data-enabled forecasting tools process large data sets to extract the essential features, predict the trends and quantify the forecasting ability. A broader impact of these tools is in addressing the need of many social and commercial systems for forecasts of future trends. In financial markets, the tool could yield forecasts for stock and other instruments, quantify the reliability of the forecasts, and predict changes in the short-term trends. In natural hazards, these techniques can be used to predict extreme events such as hurricanes, floods, earthquakes, and tsunamis from the time series data. The forecasting tools are independent of pre-determined models or parameters, and thus can provide reliable analyses of extreme events in commercial (financial markets, insurance) and social (disaster planning and management) sectors. A key need of Big Data is reliable analytic tools and the proposed data-enabled tools will address this need.
The project explored the commercialization of the research outcomes from the past NSF grant: Modeling the Multiscale Phenomena of the Magnetosphere. The main results from the past grant was the development of prediction techniques based on the dynamical systems theory. These techniques use large data sets (Big Data) and were developed for studying the multiscale nature of the Earth's magnetosphere. This led to the development of techniques for forecasting space weather and used the data-enabled approach to demonstrate its predictability. Further, the research results yielded a new approach for quantifying predictabilty in general, and thus for estimating the probabilities of the forecasts of space weather. The key feature of the new technique is its ability to extract the inherent features in the data without a priori assumptions. The general nature of the forecasting technique and its basis on the use of large data sets show its potential for application to other systems. The main goal was to apply the research outcomes of previous grants to provide reliable forecasts in financial markets, natural hazards and Big Data. A major activity of the project was the participation in the NSF ICorps curricullum and development of an approach to a proof-of-concept and prototype. The project team actively participated in the curricullum and followed up on the lessons learnt during the remaining period of the grant. The most significant activities were the close interaction with the faculty, customer interviews and feedback from the members of the cohort. These efforts were directed at developing a business plan for a start-up to commercialize the forecasting technology. The Lean Startup methodology provided a framework for the team to evaluate the different business segments for commercializing the technology. Through validated learning from the curricullum and customer interviews the financial market was identified as the best fit for the forecasting capabilities. The customer interviews in the second half of the project concentrated on this segment and the team participated in investment club meetings and meet-ups. The retail investors and fund managers showed strong interest in the forecasting tool and in back testing its capabilities using a prototype. However prototype development required more efforts and resources than was available in the project. The team continued its efforts without a prototype but it became increasingly clear that it is a requirement for advancing to the next stage of business development. A key outcome of the project is the development of a commercialization framework for the forecasting techniques in the financial sector. This requires a prototype and a plan for a prototype that addresses the unique requirements of the financial market was developed.
