This award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5). As our society stores ever-increasing quantities of digital data and abandons analog storage, we must be able to manage, organize, and preserve digital data for decades or longer. Maintaining access to large-scale digital archives will be critical in enabling future generations to access the medical records, personal information, photographs, and other data that we are generating and storing digitally. However, current approaches to archival storage are ill-suited to long-term preservation because they do not cope well with evolution of device technologies and rely upon centralized search indexes that do not scale well and are prone to failure. This project is exploring techniques that allow the management of large-scale archives containing 105?106 intelligent power-managed storage devices connected by a network. These techniques allow seamless evolution of the archival storage system by integrating new devices and removing old, inefficient devices. The research is also developing approaches to index data in the archive, allowing users to quickly find the data they need by maintaining indexes on each device and routing queries to the devices that might contain relevant data. These techniques will allow archives to scale to hundreds of thousands of devices, allowing them to contain the vast digital legacy we are leaving to our descendants. This research will guide the design of archives that are power-efficient and can gracefully evolve as storage technology changes. Additionally, the project will train both undergraduates and graduate students in the problems facing digital archiving, an area of critical importance.
