The topology of the Internet is constantly evolving and dramatic changes in end-to-end reachability have fundamentally changed the way in which malicious software propagates and is detected. At the same time, perimeter firewalls and NAT devices designed to protect networks are becoming porous to many of the threats they were designed to defend against. The end result has been a proliferation of undetected malicious activity inside network perimeters. To combat the rise of threats inside the network and the lack of visibility into sub-networks, this research seeks to construct a set of techniques for building a topologically-accurate map of unused and unreachable addresses (darknets) inside a network, and then using that map to deploy a pervasive detection system. The key insight that enables the approach is integration with routing, policy, and host management systems that already understand part of the address topology. This topology information will be used to construct a high-level map of address usage and then place darknet sensors at thousands of different locations inside the network to detect threats inside the network perimeter and threats outside trying to penetrate in. Using multi-dimensional data mining techniques we will then develop a framework for analyzing the huge volume of data produced by the detectors. In summary, this research effort introduces a novel approach to increase the visibility and effectiveness of Internet threat detection systems by developing methods to automatically discover network topology and use that knowledge to deploy pervasive network sensors that enable new detection capabilities
