The PIs proposes to bring Granite data management technologies to Grid computing to allow scientists with modest resources to work directly with n-dimensional datasets that are far too large to be stored locally. The size of scientific data sets has grown explosively in recent years, presenting new challenges for researchers and educators without local access to high performance computing resources. The vast majority of researchers cannot store datasets such as Sloan Digital Sky Survey (15 Terabytes) or Visible Woman dataset from the National Institutes of Health on local machines, but networked access to a remote data server can make valuable data available to a large number of scientists.
One of the problem areas of Grid Computing environments is a lack of tools for efficient access to multidimensional data. Currently, grid support for multidimensional data sets is largely provided using underlying scientific data APIs such as HDF5, and only recently has work begun on ways of querying remote sources of data within a multidimensional paradigm.
Intellectual Merit:
The proposal improves prefetching in environments with regular data and data access patterns by having applications encode their I/O patterns using iterators and then leveraging iterators to direct prefetching. This is a mixed approach that lies between compiler directed prefetching and predictive prefetching based on workload access patterns. The ability to extract knowledge and even to visualize the enormous amounts of data being produced daily through both observation and simulation is central to the promise of Grid computing. This capability could have significant impact on both research and education in a variety of fields, including Physics, Engineering, Medicine, and Earth Science.
Broader Impact
This project will provide support for three graduate student positions over three years. It is likely that this project will involve students from under-represented minority groups. Students will gain important research and development experience in a cutting-edge area of Computer Science. They will also attend conferences and workshops to present their own work, see the work of others, and build connections with their future colleagues.
