A new, emergent cybersecurity frontier is information manipulation, whereby adversaries may attempt to distort information to influence opinion, thought, or action. Information manipulation can take many shapes and forms, from blatant attacks such as search-poisoning, misinformation such as bogus on-line reviews, to subtle distortion such as personalized search and biased news. Left unchecked, information manipulation can harm our economy, culture, and democracy. Past research efforts have been ad-hoc and focused on specific kinds of information manipulation.
This work aims to lay the foundations of research in information manipulation. There are three research components: (1) surveying the long line of relevant scholarly work on information manipulation, and investigating how the established types of manipulation play out on the Internet and if the Internet gives rise to novel types of manipulation, (2) generating an overarching conceptual framework about information manipulation to capture the basic properties of various forms of existing and possible manipulation practices on the Internet, and (3) developing taxonomy of information manipulation that facilitates the development of general countermeasure for each category of manipulation. This project is tightly integrated with social-science research. In particular, political and social behavioral study identifies the important insights from at least two millennia of scholarship on information manipulation and help define models of the actors, goals, mechanisms, and strategies of information manipulation in the Internet era. The historical perspectives and conceptualization of information manipulation provide the foundation for developing a comprehensive framework to counter information manipulation.
A new, emergent cybersecurity frontier is information manipulation, whereby adversaries may attempt to distort information to influence opinion, thought, or action. Information manipulation can take many shapes and forms, from blatant attacks such as search-poisoning, misinformation such as bogus on-line reviews, to subtle distortion such as personalized search and biased news. Left unchecked, information manipulation can harm our economy, culture, and democracy. Past research efforts have been ad-hoc and focused on specific kinds of information manipulation. This project has laid the foundations of research in information manipulation. More specifically, the project produces an overarching conceptual framework about information manipulation to capture the basic properties of various forms of existing and possible manipulation practices on the Internet. The research activities included conducting a comprehensive survey of the long line of relevant scholarly work on information manipulation, and investigating how the established types of manipulation play out on the Internet and if the Internet gives rise to novel types of manipulation, and defining a taxonomy of information manipulation that facilitates the development of general countermeasure for each category of manipulation. This project also produces a technical framework for the detection and mitigation of information manipulation. Its data collection components include distributed measurement agents and persona generators, and its data analysis involves establishing the norms across a population and for an individual user, and detecting deviations from the population or from the user’s usual self. The framework also includes countermeasures such as profile repair or switching. This project also shows a concrete case of information manipulation where personalized search can be abused for illicit financial gains. This project develops a prototype system Bobble that is now freely available for the public at http://bobble.gtisc.gatech.edu. With Bobble, a user can compare his personalized search results with that of other users on the Internet, and identify the important difference. This project was tightly integrated with social-science research. In particular, political and social behavioral studies identified the important insights from at least two millennia of scholarship on information manipulation and help define models of the actors, goals, mechanisms, and strategies of information manipulation in the Internet era. The historical perspectives and conceptualization of information manipulation provided the foundation for developing a comprehensive framework to counter information manipulation. The research outcomes of the project have been incorporated in both Computer Science and Public Policy curricular. For example, the undergraduate Introduction to Information Security and graduate-level Network Security courses at the Georgia Institute of Technology now include topics on information manipulation. Five Ph.D. students participated in this project, and are focusing their thesis research on information manipulation.
