It is increasingly important to design cluster systems that efficiently support data-intensive computing. Parallel scientific and multimedia applications often perform large amounts of I/O either indirectly, due to swapping, or directly, due to temporary file accesses. As the disparity between the speeds of magnetic disk and other hardware such as RAM, interconnection networks, and flash continues to grow, the cost of accessing disk will increasingly become the bottleneck to system performance.
The objective of our project is to design and implement a fast backing storage system for clusters and local area networks. Our system provides a single device interface on top of multiple heterogeneous cluster-wide physical storage, particularly targeting fast random-access storage such as Network RAM and flash. Our work will free the OS from requiring multiple device-specific policies in its paging and file systems for every combination of underlying physical storage devices. As new technologies are developed, the low-level part of our system will incorporate them into its set of heterogeneous physical storage, but the OS will continue to use the same high-level interface of our system, that of a single, fast, random-access device.
The main goals of our work are: to take advantage of all types of fast storage in a cluster; to provide a single device interface that is independent of the underlying physical storage; to create a storage system that can adapt to changes in resource utilization and capacity; and to develop and examine policies for data placement and data migration between underlying devices.
