Troubleshooting undesirable network events, such as poor connectivity or performance, is difficult at best. The high-speed link techniques that work on LANs, such as dumping packets and analyzing the detailed traffic, are impossible due to massive data volume. This project will explore mathematical techniques and network tools that will reduce the amount of data that has to be captured and stored while still allowing network operators to troubleshoot their networks. The project's objective is to extract from high-speed packet streams on individual network links an approximate and highly compressed representation of the link traffic that is orders of magnitude smaller in size than the raw traffic stream but which permits almost the same degree of troubleshooting as the raw data. The project will develop the algorithms and mathematical theory needed to design intelligent sampling algorithms for compressing network traffic. Specific areas to be studied for purposes of developing sampling techniques include identifying what constitutes the representative flows for troubleshooting purposes and investigating how to best encode and decode the sampled data, and how the samples can be gracefully shrunk over time so as to reclaim space for new data as they arrive.
Broader Impact: The project will provide research experience for undergraduates. Undergraduates at Georgia Tech, Denison and other institutions will be recruited via undergraduate workshops and research symposiums. Additionally, the project will integrate education and research via inclusion of the research into courses. In addition to publishing in appropriate scientific venues, the PIs will expand the Wikipedia entries on topics related to data streaming algorithms as part of the process of disseminating general information about the topic area to the scientific community. In terms of commercial impact, the project will lead to better methods for the diagnosis of large-scale networks, thereby reducing the cost to maintain and operate them. As part of the transfer of research findings into commercial practice the PIs will collaborate with members of AT&T's Network Management and Engineering Department.
