The ongoing flood of unique data generated by satellite based instruments, such as NASA's Moderate Resolution Imaging Spectroradiometer (MODIS) systems, creates a general problem for data and archival systems that must manage it. Flowing in at roughly 1TB a day (approaching 2 PB over 4 years), MODIS data's high value requires that it be reliably preserved in long-term archival storage, yet at the same time, its current international user community wants it to remain continuously available for wide area access. The combination of massive data volumes, most of which must be stored near line, and a highly distributed user base, some of whom demand ad hoc or even interactive access, exacerbates the long standing technical and managerial mismatch between deep archival storage and wide area networks. This planning project will bring together leaders of the MODIS archive community with Computer Science researchers pioneering a new approach to wide area data management. The purpose of the MAAP project is to plan a fundable trial demonstration of a distributed service platform for providing fast, flexible, and efficient access to all MODIS archive data for a global research, education and policy planning community. In order to deal with the logistical challenges of managing massive data in networked environments, this project will explore a new common storage service, the Internet Backplane Protocol (IBP), which applies the Internet design philosophy to create a network service that can scale with the network. This technology has already been implemented in a well tested collection of middleware and has been deployed in a global testbed with over 400 nodes and 40TB of storage. A variety of application communities, including large scale content distribution, distributed data mining, and remote visualization of imaging data already use this technology. To create a successful MAAP demonstration, the team will directly engage with the scientific and management teams of the NASA-USGS Land Processes Distributed Active Archive Center (LP-DAAC).
