The importance of real-time scientific data is ever increasing, particularly in mission critical scenarios, where informed decisions must be made rapidly. Furthermore, advances in the distribution of real-time data are leading many new transient phenomena in space-time to be observed. Presently however, realtime decision-making is infeasible in many cases that require streaming scientific data to be coupled with complex models. While EarthCube will provide an unprecedented framework for disseminating data sources, the use of real-time data raises an additional set of complex challenges that must be considered. This project is a pilot to demonstrate the importance of coodinating the geosciences around their real-time data gathering and use. The work will demonstrate a Cloud-Hosted Real-time Data Services for the Geosciences (CHORDS)
The vision behind CHORDS is to provide a real-time data management infrastructure that will: a)Provide a system to archive, navigate and distribute real-time data streams via the Internet; b)Be easily deployed and configured; c)Run on cloud infrastructure; d)Use transactions built on RESTful protocols (i.e. via URLs); e)Employ data and metadata formats that adhere to standards, which simplify the user experience; f)Be free and open source.Science derived from observing platform data is the result of years of planning before a deployment, for example, and millions of dollars are spent on the deployment itself in hopes of obtaining the desired dataset. Many geo-scientific experiments are often resource constrained. In such cases it is vital to guarantee the optimal allocation of resources to ensure that important events do not go unobserved. In geo-scientific domains it is not uncommon to analyze data after a field campaigns, only to detect sensor faults, calibration offsets or anomalous behaviors when it is already too late. Real-time data will enable the optimal allocation of constrained experimental resources by automating the detection of faults and anomalies. CHORDS will provide a framework to enable adaptive sampling, discovery and fusion of various real-time geoscientific data sources, thus facilitating a new means by which geoscience experiments are carried out. The use cases will illustrate this by showing traditional experiments would have missed events of interest due to lack of access to real-time data. Focus on the initial work of defining requirements, design and specifications. Begin to ingest a small subset of geosciences data streams into a prototype CHORDS structure built in the cloud. Participate in activities that strengthen the integration of real-time data being ingested via CHORDS into other EarthCube Building Block systems that are under development. They will focus on some initial test cases, in hydrology sensor data, radar data streams, the NCAR Lower Atmosphere Observing Facilities, and outreach to earth and oceans communities.
