Scientific reproducibility -- the ability to independently verify the work of other scientists -- continues to be a critical barrier towards achieving the vision of cross-disciplinary science. Federal agencies and publishers increasingly mandate and incentivize scientists to, at a minimum, establish computational reproducibility of scientific experiments. To comply scientists must connect descriptions of scientific experiments in scholarly publications with the underlying data and code used to produce the published results and findings. However, in practice, computational reproducibility is hard to achieve since it entails isolating necessary and sufficient computational artifacts and then preserving those artifacts in a standard way for later re-execution. Both isolation and preservation present challenges in large part due to the complexity of existing software and systems as well as the implicit dependencies, resource distribution, and shifting compatibility of systems that evolve over time -- all of which conspire to break the reproducibility of an experiment. The goal of the GeoTrust project is to understand the research lifecycle of scientific experiments from conception to publication and establish a framework that will improve their reproducibility.
GeoTrust will develop sandboxing-based systems and tools that help scientists effectively isolate computational artifacts associated with an experiment, use languages and semantics to preserve artifacts, and re-execute /reproduce experiments by deploying the artifacts, changing datasets, algorithms, models, environments, etc. This reproducible framework will be adopted by and integrated within community infrastructures of three geoscience sub-disciplines viz. Hydrology, Solid Earth, and Space Science. Using cross-disciplinary science uses cases from these sub-disciplines, and engaging independent evaluators, we will assess the effectiveness of the framework in achieving reproducibility of computational experiments. Finally, verified results will be associated with ?stamps of reproducibility?, establishing community recognition of computational experiments. The framework will be developed as an EarthCube capability, with software developed and released as per EarthCube requirements. Early adopters across other geoscience sub-disciplines will be continually sought.
