This is a study of human implications of online moderation systems that deal with disruptive online behaviors, such as offensive language and hate speech, by issuing penalties such as content removal or account suspension to users they determine to be disruptive. These moderation systems usually fail to provide punished users enough support in terms of explaining why they are punished and suggesting how they can improve. Such severe limitations in fairness, accountability, and transparency lead to enormous challenges to online moderation and community wellbeing. Punished users may not understand the rationale behind penalties, and risk becoming repeat offenders. This is even more challenging when newcomers are punished for violating community norms which they were previously not aware of.

The study site is a high-population online community, where the research will document and describe human-punishment interaction (HPI) in terms of how users experience punishment, what are users' post-penalty actions, and what support resources users use for a better understanding of community behavioral standards and behavioral improvement. The research has two goals. First, empirical methods such as interview and survey will be used to investigate and theorize major sociotechnical dimensions of HPI. This will extend the existing moderation literature by articulating interactions and experiences associated with punishment and punished users. Second, more concentrated empirical methods such as narrative interview and focus group will be used to identify user-initiated ways of understanding community norms and behavioral improvement, with a focus on existing support resources that users have drawn from. Cognitive and social theories of behavior change will be used to understand existing, user-initiated ways of behavior improvement. This prioritizes a dynamic, evolving view of disruptive behavior, tracing the temporal development of user online behavior and studying and theorizing critical moments where users internalize community norms and improve behavior.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

National Science Foundation (NSF)
Division of Information and Intelligent Systems (IIS)
Standard Grant (Standard)
Application #
Program Officer
William Bainbridge
Project Start
Project End
Budget Start
Budget End
Support Year
Fiscal Year
Total Cost
Indirect Cost
Pennsylvania State University
University Park
United States
Zip Code