This project extends search engine indexing and retrieval models developed initially for structured (e.g., XML) documents to provide convenient support for capabilities required by question answering, computer-assisted language learning, and other human language technology (HLT) applications. These new capabilities are required because it is becoming common for HLT applications to use search engines to find information in large text databases. HLT applications typically use extensive text annotations to reduce the mismatch between the word-based representations convenient for text retrieval and the concept-based representations convenient for reasoning. They also can describe very specific requirements that retrieved text passages must satisfy. This use of text search is very different than ad-hoc, interactive search, and current search engines do not support it well. This project extends probabilistic indexing and retrieval models to support multiple, detailed, overlapping text annotations, hierarchical text annotations, annotations with associated confidence values, and similar capabilities. By explicitly recognizing HLT applications as first-class users, the proposed research broadens the research community's view of text search well beyond the simple queries, text representations, and retrieval models that characterize interactive, ad-hoc search today. It may also lead to improved interactive search by providing a medium in which search interfaces can express constraints and preferences derived from sophisticated user and task models. Research results are disseminated in technical papers, and as part of the open-source Lemur Toolkit. The project Web site (www.cs.cmu.edu/~callan/Projects/IIS-0534345/) provides additional information.
