As digital libraries reach petabytes in size and hold data for decades, the information that they manage becomes vulnerable to data loss from device failures and administration errors, e.g. during data migration or backup and restore. Such data loss incidents are undetectable to the user community, because the volume and lifetime of information dictates that much data are accessed infrequently or never. This project is filling a critical need for software that enables the auditing of data repositories to ensure that information is being preserved correctly. The software being developed allows digital libraries to evaluate the preservation state of current holdings across multiple, external data repositories. A new Provable Data Possession (PDP) technology allows an archive to provide cryptographically-strong evidence that it retains the exact data that were originally stored by the user community. PDP will radically change the performance of auditing data at remote sites. It allows the server to prove possession without returning data to the user. The project assembles a team of computer scientists and digital librarians developing software tools for the National Science Digital Library (NSDL). Using NSDL technologies, the team is building the Virtual Observatory (VO) Astronomy archive: a multi-institution, collaborative archive that is exploring the curation by Libraries of large scientific datasets and associated information, such as derived results, publications, and educational tools. The VO archive is being used as a testbed for the evaluation of Provable Data Possession.
