The increasing importance of ?big data?, i.e., the usage of computer systems to store and analyze large data sets, is changing the face of both science and industry. Underlying big-data analyses are big-data storage systems, including the popular Hadoop File System (HDFS). The WiS3C project seeks to re-examine the core foundations of storage systems such as HDFS, specifically by developing new specialized storage techniques to improve both the reliability and performance of these systems. The project specifically targets novel modifications in crash consistency (i.e. how storage systems keep data safe despite power loss), end-to-end data integrity (i.e., how storage systems ensure data does not become corrupt), and caching and layout (i.e., how storage systems make access to data fast); the sum of these changes will result in a new era of more robust and high-performance big-data analysis. The project will also improve the educational pipeline by connecting more undergraduates with research and attracting more women into the Ph.D. program.
