Most current NLP techniques expect input resembling read or constrained speech. When applied to spontaneous speech, such techniques encounter two serious difficulties. First, spontaneous speech contains surface phenomena relating to non-propositional aspects of the input, such as disfluencies and discourse markers. Second, spontaneous speech lacks overt punctuation for segmenting the input into meaningful units. For effective NLP, such phenomena should be overtly marked in the input; current speech recognizers, however, produce only a raw sequence of words. The goal of this project is to augment speech recognition models to output word sequences annotated for these phenomena, termed "Hidden Word-Level Events" (HWEs). New models are developed to allow recognition of HWEs to occur in tandem with word recognition. HWE recognition is based on a combination of acoustic and language models, extending the standard components found in current systems. The new models also capture prosodic characteristics of HWEs, including intonation and duration patterns. The prosodic information is combined with statistical language models describing the distribution of HWEs in relation to lexical and syntactic units. Results should significantly enhance our ability to process spontaneous speech automatically; the research will also further our basic understanding of spontaneous speech.
