Science and engineering are increasingly relying on data and the ability to process a massive amount of data to solve hard problems and drive fundamental discoveries and innovations. Challenges arising from this trend are often referred to as "Big Data" problems. Examples of big data processing applications include seismic data analysis, data-intensive text processing, assembly of large genomes, machine learning, data mining, and social-network analysis.
This project will investigate new directions in software defined networking (SDN) that are motivated by the networking challenges stemming from big data processing applications and by the potential benefits of using optical lightpaths for big data transport. The project will develop effective solutions for jointly configuring a rich set of optical devices and SDN switches to realize network services that meet the needs of big data applications. Specifically, the project will develop optical device resource allocation algorithms, topology design and routing algorithms, comparisons between greedy and guaranteed resource allocation policies, co-scheduling systems for traffic and network, techniques for data shuffle transmissions, and co-designed application and network controllers. The approaches, algorithms, and software developed by this project will be evaluated in a realistic experimental infrastructure called BOLD.
The project may have far reaching societal impacts beyond the computing discipline. Results from the project can dramatically speed up a wide range of computational scientific discoveries. Optical networking devices consume very little power, yet can support enormous data rates; the project results could lead to a more environmentally sustainable future for the IT industry. The project activities will provide exciting opportunities for training and education of undergraduate and graduate students, and particularly under-represented minority students, in cutting edge big data-driven networking. Finally, software, data, and curriculum materials produced by this project will be disseminated as free open-source resources for the wider community's use.
