The human ability to use language flexibly is a hallmark of robust intelligence. In interactive dialog, utterances are dynamically tailored to the common ground or specific context with specific partners. However, interaction with spoken dialog systems is highly constrained and constraining, allowing speakers very little flexibility in what they can say while the system presents pre-determined messages. To make interactive dialog technology broadly useful, this exploratory interdisciplinary project collects a corpus of dialogs exhibiting some important sources of variation, analyzes the corpus, and uses the resulting analyses to develop models and prototype implementations of dynamic dialog strategies. The ultimate goal of this effort is to support the synthesis of entirely new, flexible, and robust spoken dialog systems that are capable of adapting on-line.
The Walking-Around corpus consists of 40 human-human dialog interactions where a remotely located person gives directions to a pedestrian walking around in an urban or campus environment. The experimental paradigm varies the friendship relationship of the dialog partners, whether the director can see what the pedestrian sees, and the familiarity of both the director and the pedestrian with the environment. No other existing direction-giving corpora model dialog interaction in an outdoor real-time environment where the physical context grounds the dialog context. The resulting corpus is used to test hypotheses about, and develop models of, the evolution of local and global dialog adaptation strategies. Key to our effort is determining which adaptations are actually functional, that is, beneficial for a particular task or context in spoken dialog systems.
This project develops a better understanding of the processes of entrainment in dialogue and extends our ability to computationally model it. We developed a dialogue system that dynamically entrains to the user, using novel types of entrainment. Our specific activities were: Corpus Collection: We collected dialogue corpora in two different physical situations exploring different aspects of "entrainment in the wild", in natural conversational settings where people are walking around in the physical world, outside of the lab. These corpora are available to the research community. Psycholinguistic modeling: We developed a number of psycholinguisic models of entrainment Computational modeling: We model entrainment on the collected corpora using automatic methods and building on hand annotations. This allows us to model entrainment of lexical and syntactic structures beyond referring expressions Computational prototype dialogue system: We implement working models of entrainment with a spoken language generator and tested them with users in a user study. We show that a dialogue system with entrainment is more natural and more friendly than one without. Our results and algorithms can be used in development of new dialogue systems for many applications of broader impact, such as education, assistive and well being dialogue systems. Dialogue systems that adapt to the user have been shown to improve compliance for medically recommended exercises, to be easier to use.
