Solid-state data storage built upon NAND flash memory is fundamentally changing the memory and storage hierarchy for virtually the entire information technology infrastructure. Nevertheless, there have been several fundamental and challenging issues to be addressed before the industry can explore the flash memory to its full potential. First, as flash memory technology scales down, its reliability degradation approaches to an alarming level, leading to serious concerns and skepticism of storage system architects and users in many applications. Second, system and application development of solid-state storage has been independently conducted, resulting in isolation, duplicated operations, and an inefficient management among these layers. Due to the technology scaling and information loss in existing simple interface with storage devices, flash memory has not been efficiently and reliably utilized in practice, and the situation will become worse with the technology scaling. The PIs of this project will apply a holistic system design methodology to cohesively address the challenges preventing wider adoption of flash memory. By innovating well-orchestrated cross-layer information sharing and utilization, this design methodology enables seamless utilization of system-level workload and physical-level device characteristics across the entire software/hardware stack without complicating overall system design. An integrated software and hardware prototyping infrastructure will be developed to demonstrate the potential using major and widely used software systems, such as Hadoop, virtual machines, and database. This project will achieve a high broader impact by transforming basic research results into storage systems, and by training both undergraduate and graduate students with research activities, and by timely integrating new research results to classrooms.
Specifically, this project will carry out several closely related tasks: (1) It will develop techniques that can learn and predict the varying characteristics and their correlations of individual flash memory devices. This will provide run-time information that makes it possible to optimize the use of flash memory for alleviating the reliability crisis and adapting to varying system-level workload characteristics. (2) It will develop techniques that enable critical information exchange across the storage hierarchy in order to facilitate cross-layer information sharing. (3) It will further develop a set of techniques across the design hierarchy that can effectively utilize these runtime collections and predictions to improve the overall system reliability and performance.
