This is a Research Initiation Award (RIA). The research aims at resolving long-standing problems with the use of speech-act theory in computational systems for natural language understanding and to provide methods of interpreting unconstrained discourse. The integration of verbal and non- verbal speech acts are still very poorly understood and is critical for human-computer interactions. This research seeks to integrate many modalities with the study of speech acts, an important conceptual framework in speech and dialog processes. The goals are: (1) to determine how the theory of meta-acts can be extended and refined to model negotiations of reference and confirmation of mutuality of knowledge, and (2) to determine if meta-act models can be validated predictively. Meta-knowledge models include information about conversations that are non- verbal, such as turn-taking. The project derives meta-acts as a rational strategy of action, given attributions of the participants' beliefs, goals and expectations at the point in the dialogue in which the acts actually occur. The acts are then expressed in a rule-based simulation of similar conversations. A byproduct of the research is a database of analyzed and transcribed data useful for other researchers. It is expected that there will be substantial impact in human-computer interactions research and aspects of research in speech and natural language processing.
