Machine learning techniques, particularly deep neural networks, are increasingly integrated into safety and security-critical applications such as autonomous driving, precision health care, intrusion detection, malware detection, and spam filtering. A number of studies have shown that these models can be vulnerable to adversarial evasion attacks where the attacker makes small, carefully crafted changes to normal examples in order to trick the model into making incorrect decisions. This project's goal is to develop formal understandings of and defenses against these vulnerabilities through characterizing the relationship between adversarial and non-adversarial examples, developing mechanisms that exploit this relationship to support better detection of adversarial examples, and metrics and methods to demonstrate the robustness of machine learning models against them. Together, the theories, algorithms, and metrics developed will improve the robustness of machine learning systems, allowing them to be deployed more securely in mission-critical applications. The team will also make their datasets and source code publicly available and use them in their own courses and research with both graduate and undergraduate students, with particular efforts to include students from underrepresented groups in Science, Technology, Engineering and Math. The work will also support high school outreach programs and summer camps to attract younger students to study machine learning, security, and computer science.
The project is organized around three main thrusts that combine to provide a holistic approach to modeling and defending against evasion attacks. The first thrust aims to characterize both normal and adversarial examples via systematic measurement studies. This includes considering different types of regions around specific examples (e.g., metric ball, manifold, and transformation-induced regions) and then characterizing the examples' vulnerability based on a number of algorithms for combining classifications of other examples in the nearby regions. The second thrust focuses on designing robust defenses against adversarial examples by using representative data points in a region, aggregating multiple data points, and using a diverse set of classifiers to reduce the vulnerability induced by using single data points or algorithms. The third thrust involves defining metrics for modeling robustness along with theories and algorithms that leverage those metrics to analyze model robustness. These include lower bounds of adversarial perturbation in metric balls, robustness metrics based on computational costs, analyses of the representativeness of new datasets relative to training data, and methods for leveraging human estimation of adversarialness.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
