A central problem in network management is how to monitor, analyze, and react to changing network conditions due to various reasons such as failures, misconfigurations, routing instability, flash crowds, distributed denial-of-service (DDoS) attacks, and computer worms. Manually tracking and reacting to such changes is labor-intensive, error-prone, and in many cases infeasible (e.g. modern worms can causecatastrophic damages within minutes -- too small a timescale for human to respond manually). It is therefore an important and urgent research challenge to develop tools and techniques to fully automate the process of network monitoring, analysis, and response. The research addresses the above challenge by developing a ScalableMonitoring, Analysis, and Response Toolkit (SMART). SMART allows applications to perform network-wide traffic monitoring, identifyanomalies and changes in the network state, diagnose the causes for the changes, and dynamically control the network traffic -- all in an automated fashion. By integrating practical network domain knowledge and engineering experience with solid theoretical foundations in algorithms, statistics, data mining, and machine learning, SMART is scalable, accurate, robust, and easy-to-deploy. It can significantly simplify the development and deployment of large-scale network management and security applications. To demonstrate the practical values of SMART, the research develops two significant real-world security applications using the toolkit: automated worm fingerprinting, and network-based DDoS defense. Finally, the research includes a significant education and training component. The research results of the project will be integrated into both the undergraduateand the graduate curricula through the development of new courses.
