This project will examine whether federal agency rulemaking can be improved with two innovations: a) multi-level deliberation (MLD), in which people discuss rulemakings in small groups that then select members to represent the group in a higher-level group and b) the combination of language technologies into an artificial discussion facilitation agent (DiFA). The project poses computer science challenges of combining several Natural Language Processing technologies (primarily Interactive QA, Dialogue Analysis, and Summarization) into a viable facilitation agent and in applying these technologies in an eclectic, multi-user discussion environment. We expect advances to be made within each component technology. For example, we hope to increase the utility of Dialogue Act tagging across applications and domains by using a set of general discussion tags for tracking and summarizing threads of discussion by combining dialogue structure and content analysis. We will also investigate how general our Question Answer approaches are. The social science herein breaks new ground in the nascent fields of e-rulemaking and democratic deliberation research. The project will advance research on measuring the quality of deliberation and the effects of deliberation and DiFA on individuals and communities. Research will involve four rulemaking experiments. The first three are subsets of the final one. The final 3X2 experiment crosses MLD, non-MLD deliberation, and non-deliberation with the presence or absence of DiFA. The success of the various conditions of these experiments will be measured using a multi-trait, multi-method approach that will include survey and focus group measures of agency official and participant perceptions and evaluations, a content analysis measure of the cognitive sophistication of rulemaking comments, both human-coded and automated content analyses of the quality of deliberation, measures of the impact of the deliberations on participants (knowledge, trust, citizenship), DiFA usage patterns, and continued participation in our user community.
The federal agency rulemaking comment process represents an important potential avenues by which the American public can affect how it is governed. Such comments can make agency officials aware of likely adverse effects of the proposed rules. Unfortunately, the current rulemaking comment process faces a number of social and organizational problems including poorly informed and distrustful participants, lack of dialog among participants that could sharpen their reasoning, and problems of scale such as the large number of comments generated. Researchers believe that most rulemaking comments are low in quality or redundant?a product of form letters used by public interest groups. Rulemaking is not a plebiscite, but an effort to identify reasons to accept or modify proposed rules. This project will seek to address the problems of existing rulemaking by immersing rulemaking participants in small discussion groups that will be assisted by discussion facilitation software. The software will use cutting-edge technologies to help answer questions, summarize discussion, and provide feedback and suggestions on their discussion. Discussion itself will be organized into a hierarchy of representative groups to help the best ideas spread among participants and rise to the top. The value of the technology and of the deliberation methods will be thoroughly tested using experimental methods and data collected via surveys, focus groups, and by the software. The project will advance research in several areas. In computer science, it will seek to apply natural language technologies in a more general setting than before. The technology created could have broad application. It will also combine several technologies into a discussion facilitator that may be more widely used. The project will also advance research on democratic deliberation by improving and testing measures of deliberative quality and by adding to knowledge of how deliberation affects citizens.
