This project entails research and development and to address the software and tools problems fro ultra-scale parallel machines, especially targeted for scalable I/O, storage and memory hierarchy. The fundamental premise is that to achieve extreme scalability, incremental changes or adaptation of traditional (extension of sequential) interfaces and techniques for scaling data accesses and I/O will not succeed, because they are based on pessimistic and conservative assumptions of parallelism, synchronization, and data sharing patterns. We will develop innovative techniques to optimize data access that utilize the understanding of high-level access patterns ("intent"), and use that information through runtime layers to enable optimizations and reduction / elimination of locking and synchronization at different levels. The proposed mechanisms will allow different software layers to interact/cooperate with each other. Specifically, the upper layers in the software stack extract high-level access pattern information and pass it to the lower layers in the stack, which in turn exploit them to achieve ultra-scalability. In particular, the main objectives of this project are: (1) Techniques, tools and software for extracting data access patterns and data-flow at runtime; (2) Interfaces and strategies for passing access pattern across the different layers for optimizations; (3) Implementation of these techniques in appropriate layers such as parallel file system, communication software 9e.g., MPI2), and runtime libraries to reduce or eliminate synchronization and locking; (4) Runtime techniques and tools that exploit access patterns for reducing power consumption and cooling requirements for the underlying storage system; and (5) Development of interfaces and software to use active storage for data analysis and filtering
