Institution: National Center for Atmospheric Research Abstract date: 12/19/06
Moore: 0632313
This project will develop a Cooperative Arctic Data and Information Service (CADIS) that will support the Arctic Observing Network (AON) and Study of Environmental Arctic Change (SEARCH) programs. CADIS will provide the discovery, access, and use of scientific data by providing near-real-time data delivery, a repository for data storage, a portal for the discovery, and tools to manipulate data. This system and data service will be built in a stepwise coherent manner and result in comprehensive long-term management for Arctic scientific data. CADIS will be a joint effort of the University Corporation for Atmospheric Research (UCAR), the National Snow and Ice Data Center (NSIDC), and the National Center for Atmospheric Research (NCAR). The project team will develop a new body of cyberinfrastructure by leveraging, integrating, and extending UCAR's Community Data Portal (CDP) framework and Unidata's THREDDS environment to form the CADIS system and portal. The CADIS portal will make it easier for scientists to locate, display, subset, publish, and analyze related data sets provided by a network of data providers. In the first year, a metadata plan will be completed; it will include AON projects and Long Term Observatory (LTO) projects. In the second year, the CADIS portal will be populated with metadata from the AON and LTO projects. In the third year, real-time delivery through CADIS of selected AON and IASOA (International Arctic System for Observing the Atmosphere) data will be accomplished, and tools for searching via a map interface, and a map server showing the location of selected AON or SEARCH components (where metadata are available) will be added. Also in the third year, system performance will be evaluated and documented, and a future direction charted. Guidance from the scientific and lay user communities will be key to implementing the CADIS facility. Information will be received via questionnaires, meetings, standing committees and individual queries to assess CADIS effectiveness and recommend improvements. This project is highly relevant to International Polar Year goals for developing comprehensive data management plans and creating legacy data sets. The intellectual merit lies in the stepwise development of a new cyberinfrastructure for management of Arctic scientific data. The broader impact of CADIS is that it creates a foundation for long-term access to data archives, discovery, delivery and analysis by the Arctic science community and other users.
Overview The Arctic Observing Network (AON) is composed of a group of about 40 investigations aimed at making observations of Arctic environmental conditions through a period of major changes in the Arctic weather, ice, and climate conditions. AON was envisioned as "…a system of atmospheric, land- and ocean-based environmental monitoring capabilities--from ocean buoys to satellites--that will significantly advance our observations of Arctic environmental conditions". Its data service was envisioned as a portal through which scientists can archive and share their data; find all data relevant to a location, project, or process; browse imagery and complete documentation; visualize data; and query multiple datasets and sources. In 2007, the National Science Foundation (NSF) funded the National Snow and Ice Data Center (NSIDC), the National Center for Atmospheric Research (NCAR), and the University Corporation for Atmospheric Research (UCAR) to develop the Cooperative Arctic Data and Information Service (CADIS) to accomplish the above objectives. See: http://aoncadis.org/. We found that interdisciplinary data discovery, a central theme of AON data management, was challenging. One of the major CADIS accomplishments was to provide the community a single place to archive data and metadata (information that describe data) and to search, access, and display all the data from AON. Another challenge in the design of CADIS was to make sure the system was flexible enough to handle the exceptional disciplinary diversity of AON data for long-term data stewardship. CADIS Impacts, Lessons Learned and Contributions 1. One size does not fit all. Although it is a worthy and perhaps necessary long-term goal to ask all investigators to submit their data using certain standards and conventions, it has not been feasible due to the wide diversity of the data and different ways they are recorded and analyzed. 2. The CADIS has a combination of data, documentation, and metadata from AON. In some instances, the actual data resides with the investigators at their home institutions, but they can be found through CADIS. The CADIS metadata database is the key to assuring the discovery and long-term data access by the broad science community and automated sharing of AON data across distributed archives. The most significant lasting accomplishment of CADIS may be the development of an easy to use metadata generation tool. 3. Collaborative and consensus-based decision making efforts and the development of a shared vision for a multi-organization project take time and sustained, well-sponsored effort. The success of CADIS was possible because NSF was able to provide stable funding to allow the development and implementation of CADIS. 4. The CADIS team was a diverse group of professionals with different perspectives all working together to develop this new paradigm for NSF data management support. The mix of experiences in all aspects of data management proved very beneficial to the group as it balanced the challenges of implementing the latest technology with the reality of supporting individual data providers from various different science disciplines. 5. Projects should not be over engineered and should evolve organically and adapt to change. The original design of CADIS, written in the proposal 5 years ago, changed over time as the CADIS team analyzed and implemented new capabilities and adapted the system to meet the needs of the AON community. 6. Successful projects both lead and follow, for example, being responsive to community needs while showing leadership and innovation. It is important to seek out and engage the community during the development of such a system and continue to seek input as the service is deployed and the project evolves. 7. The CADIS has been more successful with the provision and sharing of data, and much less so in providing tools or getting the community to adopt its tools. Scientists have their own "toolkits" and workflows and they are reluctant to change. Considerations for the Future The CADIS has helped position the Arctic community to meet the new national priority for data stewardship and, in particular, the new NSF requirements in this area. The framework of CADIS and the focused user support will help the investigators meet these requirements. There is an opportunity in the future to unify the diverse data holdings across other Arctic projects to simplify access and improve the ability of the broader community to work with these rich datasets. The CADIS framework can be customized to meet the needs of other projects requiring a well organized data management strategy, implementation and support structure. This is only possible with a well-organized team of people to monitor and manage the system; a support function to help data providers and users; developers who take advantage of new technologies and strategies; and longer term stable resources for accomplishing the tasks described above.
