Making changes to the established Internet architecture and its core protocols has become increasingly difficult as the Internet has exploded in popularity. Changes may require modifying dozens or hundreds of software implementations deployed on hundreds of millions of computers. Other than sheer scale, a major reason that these changes are difficult to deploy is a high degree of coupling between applications and the underlying network protocols and their semantics. This coupling is evident for an important class of network applications that function primarily as data transfer agents (e.g., email, file transfer, web, and so on). These applications typically bundle the functions of content negotiation---naming objects, negotiating common languages, handling cookies, etc.---and data transfer---the process of actually sending the data across the network. The result of this coupling is that in order to deploy a novel data transfer mechanism, one must modify every application that makes use of the network.
Intellectual Merit: The Principal Investigator proposes to design and implement a novel data-oriented transfer layer that performs data transfer on behalf of applications. The transfer layer will be a narrow waist at which to introduce new mechanisms for data transfer, and have them immediately benefit all applications using the transfer layer. By serving as a locus for modifications, the transfer layer will also facilitate the adoption of changes within the underlying network. A new transfer-layer module will be able to make the network-intensive portion of applications use new network-layer functionality with no application modification.
As a first use of the transfer layer, this project will develop novel techniques for improving the resilience of Internet applications to challenging network environments. These environments are characterized by intermittent connectivity or poor link quality, such as that provided by cheap wireless links or satellites. The techniques developed will be used to create a network that performs drastically better in the face of accidental and malicious failures. The approaches the project will explore include performing transfers by using multiple network paths in parallel, supporting out of band transfers via portable storage, and ensuring that transfers continue to make forward progress in the face of network interruptions.
This project will address several major challenges to building and deploying a practical transfer service. The first is the design and evaluation of the interface that the transfer service provides to applications. The design must simultaneously fit naturally into existing and future applications, while not precluding the optimization necessary for its use in a production environment. The second challenge is the development and implementation of a security model that balances the need to provide equivalent or better security while allowing an efficient implementation. The appropriate security model will likely require the development of novel techniques for securely sharing cached data. The third challenge is the design of algorithms to select among the different sources of data that the transfer layer will enable. These algorithms must react to changing network conditions and balance between shared paths with incredibly different performance characteristics.
Broader Impact: Continued evolution is critical for making the Internet and its protocols suitable for more applications, particularly those that require more availability than today's Internet can provide. Improved protocols may also significantly reduce the operational costs associated with managing the Internet. This research may significantly accelerate the rate of adoption of new protocols and technologies by providing a narrowed interface on which developers can concentrate, instead of spreading their effort across dozens of different applications. The techniques developed as a part of this research for communication in challenging environments could help improve the availability and reach of the Internet into areas formerly only poorly served by network connectivity. The project will release a freely available reference implementation of the transfer service for others to build upon.
The PI will use the framework developed in the course of this research as a central part of an upcoming graduate class at CMU. If successfully deployed, the transfer framework will be used as a vehicle for making course content (slides, notes, assignments, and videos of each lecture) from the PI's classes freely available on the Internet in a way that is accessible to students in underserved areas with poor network connectivity.
This grant sought funding to explore the design, implementation, and evaluation of a novel "data-oriented transfer layer" for Internet applications. This "transfer layer" would perform data transfer on behalf of applications: Where today, each application (such as a web browser or email reader) implements its own mechanisms for data transfer, this project explored the benefits that could be obtained by having all applications running on a computer share a service that would perform such data transfers on their behalf. The potential benefits of this architecture were increased flexibility and evolvability (existing applications could take advantage of improvements in data transfer techniques added to the transfer service); the development of innovative mechanisms for data transfer; and the ability to increase efficiency through cross-application sharing when multiple applications request the same or similar data. The project had several positive outcomes. Beyond the intellectual merit, development and release of usable software, and identification of several networking problems of substantial commercial importance, the research conducted within this project formed an important part of three students' Ph.D. theses; one of these students is now CEO of a computer storage systems startup, one is a researcher at IBM, and one is at Microsoft. The project also involved collaborations with colleagues in computer science theory, and led to a collaborative research effort in optical networking with colleagues at Intel and UCSD. The core hypothesis of the project appears supported: Providing a common set of tools for data transfer provides substantial benefits to evolvability. This core idea is being explored in more depth today by several projects in the networking community in the form of "content-centric networking". Along with this architecture, the project developed several innovative mechanisms for how to accomplish data transfer. Several explored new mechanisms for peer-to-peer data transfer, such as "SET" -- Similarity Enhanced Transfer. In typical P2P data transfers, "peers" can only download from other nodes that have the exact file they are interested in. SET extends such systems to allow peers to download from "similar" files. For example, a SET peer might be able to transfer much a powerpoint presentation from a nearby computer that had an older version, and retrieve only the newer parts from a farther-away node. Together with a colleague expert in storage systems, the project identified a critical problem facing datacenter networks during high-bandwidth, high-fan-in data transfer, called the "Incast" problem. This problem is similar to what happens when several people all try to talk to one listener at the same time: Some of the data gets lost, causing slowdowns and unpredictable speeds. The problem is particularly important at the huge scale now being achieved by industry giants such as Google, Microsoft, and Facebook. The work done under this project spurred a large amount of follow-on work, with over 70 papers written so far that build upon it. Overall, the PI believes that this project was successful on technical grounds, and in meeting the program's goals of building ``a foundation for a lifetime of leadership'' in both education and research. The CAREER award was one of the PI's first major awards, and it proved instrumental in supporting research that led to several additional research programs, and directly contributed to the PIs continued involvement in the Future Internet Architecture program. .adslot-overlay {position: absolute; font-family: arial, sans-serif; background-color: rgba(0,0,0,0.65); border: 2px solid rgba(0,0,0,0.65); color: white !important; margin: 0; z-index: 2147483647; text-decoration: none; box-sizing: border-box; text-align: left;}.adslot-overlay-iframed {top: 0; left: 0; right: 0; bottom: 0;}.slotname {position: absolute; top: 0; left: 0; right: 0; font-size: 13px; font-weight: bold; padding: 3px 0 3px 6px; vertical-align: middle; background-color: rgba(0,0,0,0.45); text-overflow: ellipsis; white-space: nowrap; overflow: hidden;}.slotname span {text-align: left; text-decoration: none; text-transform: capitalize;}.revenue {position: absolute; bottom: 0; left: 0; right: 0; font-size: 11px; padding: 3px 0 3px 6px; vertial-align: middle; text-align: left; background-color: rgba(0,0,0,0.45); font-weight: bold; text-overflow: ellipsis; overflow: hidden; white-space: nowrap;}.revenue .name {color: #ccc;}.revenue .horizontal .metric {display: inline-block; padding-right: 1.5em;}.revenue .horizontal .name {padding-right: 0.5em;}.revenue .vertical .metric {display: block; line-height: 1.5em; margin-bottom: 0.5em;}.revenue .vertical .name, .revenue .vertical .value {display: block;}.revenue .square .metric, .revenue .button .metric {display: table-row;}.revenue .square .metric {line-height: 1.5em;}.revenue .square .name, .revenue .square .value, .revenue .button .value {display: table-cell;}.revenue .square .name {padding-right: 1.5em;}.revenue .button .name {display: block; margin-right: 0.5em; width: 1em; overflow: hidden; text-overflow: clip;}.revenue .button .name:first-letter {margin-right: 1.5em;}a.adslot-overlay:hover {border: 2px solid rgba(58,106,173,0.9);}a.adslot-overlay:hover .slotname {border-bottom: 1px solid rgba(81,132,210,0.9); background-color: rgba(58,106,173,0.9);}a.adslot-overlay:hover .revenue {border-top: 1px solid rgba(81,132,210,0.9); background-color: rgba(58,106,173,0.9);}div.adslot-overlay:hover {cursor: not-allowed; border: 2px solid rgba(64,64,64,0.9);}div.adslot-overlay:hover .slotname {border-bottom: 1px solid rgba(128,128,128,0.9); background-color: rgba(64,64,64,0.9);}div.adslot-overlay:hover .revenue {border-top: 1px solid rgba(128,128,128,0.9); background-color: rgba(64,64,64,0.9);}
