End-to-end data transfer rate requirements in the physics and astronomy communities are soon to approach the terabit-per-second regime. Unfortunately, the state-of-the-art in end-to-end transport protocols scales to at most a few gigabit-per-seconds of single-stream steady-state throughput. This project considers the novel paradigm of "packet-scale congestion-control", which can potentially improve scalability by several orders of magnitude. The paradigm, however, faces three fundamental issues that seriously challenge its promise to deliver in practice. This project addresses these risks by: (i) Developing formal queuing-theoretic models for the packet-scale paradigm and studying and addressing the impact of "noise" in end-to-end feedback. (ii) Conducting closed-loop analysis of interaction of the paradigm with network elements to study stability, fairness and burstiness in aggregated settings. (iii) Designing OS mechanisms that can ensure accurate inter-packet spacings and robustly deal with inaccuracies. This project is expected to lead to transformative innovation in both formal methods as well as end-host implementations that are likely to be key ingredients for future ultra-high-speed transport protocols. Such protocols will be significant enablers for distributed scientific computation by the High Energy Physics, Bio-informatics, and Radio-astronomy communities. Second, ultra-high speed protocols that can run without adverse effects on the shared public Internet will result in enormous infrastructural cost-savings. Third, the project will be an excellent source of undergraduate and graduate students trained in experimentation, measurements, and scientific analysis---skills that are invaluable for many federal, commercial and academic institutions. Fourth, through involvement of minorities, it will help broaden the diversity of the Computer-Science work force.
The main focus of this project was to evaluate the practicality of the Packet-Scale congestion-control paradigm for large data transfers in ultra-high speed networks. For this, it has to address three main risks that challenged the paradigm in the real-world---stringent end-system support, sensitivity to real-world noise, and behavior in an aggregated environment. The project made significant accomplishments towards its goals. Intellectual Merits: The reasearch conducted in this project has made significant leaps in the design of congestion-control protocols---shedding the legacy framework of RTT-scale design allows the packet scale paradigm to fundamentally address the issues of RTT unfairness, the speed-overhead tradeoff, as well as co-existence with conventional Internet traffic. By desiging a working Linux implementation of the paradigm and extensively evaluating it in real-world testbeds, this project has confirmed that the impact promised in simulations, can indeed be translated to the real-world to a surprising extent! A second fundamental contribution of the project is its ability to successfully alleviate the impact of noise on the paradigm. Noise is an issue that plagues not just the paradigm, but all of the past literature in the related fields of high-speed bandwidth estimation, as well as delay-based congestion-control. The networking research community has remained by-and-large suspicious of the scalability or practical value of these fields due to their sensitivity to the presence of real-world noise. By alleviating the impact of noise within the pcket scale framework (which is impacted by it to a much lager degree than any of these related field), the project has taken a significant step in convincing the research community that the related fields are also worth giving a serious consideration. A third significant impact of this project is in the field of operating system design. The project has demonstrated that using software-only techniques within modern interrupt-driven operating systems, it is possible to implement the packet scale paradigm, such that it can work well at speeds of up to 10 Gbps. We expect that as the paradigm gets adopted more widely, some of these techniques (for gap creation and packet timestamping) that have been designed in this project will be integrated in future operating systems as primitive components that can be used by other protocols as well as resource-management frameworks. Broader Impacts: The project accomplishments benefit several "big data" communities. The availability of an implementation of an ultra high-speed transport protocol enables the use of the wide-area networks for scientific computation and distributed analysis by the Computational Biology, High Energy Physics, Bio-informatics, Environmental Modeling, Radio-astronomy, and Health Informatics communities. Furthermore, the fact that implementation is available as a loadable kernel module (that also works with legacy TCP applications as-is) goes a long way in easing the hesitation of system administrators within these scientific networks, in installing and trying out the paradigm. Second, an ultra-high speed transport protocol that can co-exist with conventional Internet transfers, without adversely affecting the performance of the latter, results in enormous infrastructural cost-saving—scientific research communities that use the paradigm can now use existing public networking infrastructure worldwide, without continuing to deploy dedicated networks. This has remained impossible to do with any other implementation of a high-speed transport protocol. Third, experience in prototyping, experimentation, measurements, and scientific analysis is invaluable to federal, commercial, and academic institutions that are involved in mining for information in large data-sets. This project has helped train 6 graduate students (including those from under-represented communities) and 1 undergraduate student in these aspects.
