The purpose of this project is to develop probabilistic language models that efficiently integrate denotational semantic information into syntactic and phonological stages of recognition for use in spoken language interfaces to sensor or robotic agents. This integration is intended to allow information about the meanings or denotations of words in the agent's current environmental context to influence the probability estimates it assigns to hypothesized analyses of its input, before any recognition decisions have been made, so that the interfaced agent can favor in its search those analyses that "make sense" in its representation of the current state of the world. Since these models can be trained on the same kinds of examples that may be used to establish word meanings in the agent's lexicon, it is expected that they will be easier to adapt to changing domains than those relying exclusively on word co-occurrence statistics in fixed corpora.
A recognizer based on this model will eventually serve as a testbed for evaluating spoken language interfaces to networked scout robots in search and rescue applications and mobile manipulation robots in home care tasks or materials handling applications in joint projects with other members of the Artificial Intelligence Robotics and Vision Lab, as well as in applications that require navigating digital road maps or other geographic data in joint projects with members of the Spatial Databases group at the University of Minnesota. This model will also be adapted as a framework for teaching natural language processing concepts, providing students with a broader context in which various processing components may fit together, and inviting students to consider innovative solutions to natural language processing problems that cross the boundaries among traditional components.
