Large-scale botnets have become a blight on the Internet. Botnets engage in a variety of harmful activities, including initiating DDoS attacks, committing click fraud, propagating adware, and sending enormous volumes of spam. Though there is an increasing awareness of botnets, there are gaping holes in our understanding of botnets, both in terms of macroscopic properties as well as the ability to track and thwart specific attacks.
As part of this project, we develop a response to the botnet threat by building a monitoring system that gathers and distributes objective data on the problem. Our work offers three novel contributions. First, we solve many of the challenges involved in building a real-time botnet monitoring platform. For example, our system executes live botnet nodes, and as such, it must prevent these nodes from causing harm to other hosts on the Internet. Second, we implement several prototype defensive tools that take advantage of the real-time information provided by the platform. Third, our work exposes the rich texture of the botnet ecosystem by analyzing botnets from multiple perspectives and by correlating the attack vectors with observations of real bots executed in laboratory settings.
Our botnet monitoring platform thus advances our understanding of botnets and enables promising anti-botnet defense tactics. It thus serves as a crucial step in the development of a trustworthy network that can support a much wider diversity of uses than can be found on today's Internet.
Botnets are a growing threat. Massive-scale distributed botnets are an unfortunate everyday reality of the Internet, and are the basis of a lucrative and difficult to detect underground economy on the Internet today. They steal identities and financial information, send most of the world's spam, generate phishing campaigns, propagate worms and trojans, and are used in DoS protection rackets. Further, as demonstrated recently with the attack on Estonia, a large botnet, properly controlled and targeted, can disrupt virtually any mission critical use of the Internet. Though there is an increasing awareness of botnets, there is a dearth of detailed knowledge regarding their activities especially in real-time. This includes both network-wide cumulative statistics such as estimates of number of compromised hosts as well as more detailed information such as which botnets are active currently, what attacks are being initiated by them, what compromised hosts belong to a certain botnet, and so on. Such knowledge is crucial for many tasks, such as identifying and reclaiming compromised nodes, safeguarding users from botnet activity, and reacting to attacks at their very onset. As part of this project, we have developed a significant response to the botnet threat by building a monitoring system that can be used both to understand the nature of botnets and to gather and distribute objective data on the problem. Our research leverages previous work on malware measurements and recent advances in VM execution, but also addresses a number of other additional technical challenges. One primary challenge is scalability that manifests itself in different forms. Online analysis of high volume spam streams required innovative stream processing techniques. Crawling of potential sources of malware, controlled execution of malware to separate bot binaries from other forms of malware, and participating in large numbers of botnets all required new techniques to prune the search space and scalably execute large numbers of malware classified as bots. Another challenge addressed is how to execute live bots and monitor its activities without contributing to its attacks. Overall, we feel our work advances the understanding of botnets and enables promising research in anti-botnet defenses. An essential step in the success of the project is external adoption of our developed tools, and thus we have been working with industry to achieve the same. Our work on SearchAudit and deSEO has been transitioned into the security groups associated with Microsoft Bing and robust, commercial implementations of our techniques are currently under development. The BotLab system has been effective at gathering objective data regarding spamming botnets, and this data is being used by other researchers and the FBI to combat the botnet threat. For instance, the security research groups at UCSD and UMich have used the dataset to perform further analysis and to combine it with other data obtained through different sources. Botnets have been a growing threat in the last decade, and there have been individual measures to fight them, but there has been no comprehensive monitoring system that can provide up-to-date information on botnets. In this project, we have examined botnets and other malware prevalent on the Web, and we have developed and deployed a real-time botnet monitoring platform that is comprehensive, tracking both botnet activity and methods of propagation employed by them. We have also developed tools to detect malicious activity on the Web and have explored techniques to curtail the growth and spread of botnets. In particular, we have shown that it is possible to build a system, which can, in a timely fashion, with minimum human interaction, track and monitor botnets--- their communications, their activities, and their propagation. By building such a system, we have helped shed light on botnet operations, thereby improving our collective understanding of botnets. We have built infrastructure to measure botnet activities and communication, and have also built several tools to understand and defend against the various steps involved in botnet propagation. Website: http://botlab.org/
