This Small Business Innovation Research (SBIR) Phase I research project aims to make trend discovery on text documents a mainstream activity within the familiar workflow of search engine users, by using a linguistic approach that leverages many years of prior commercial research and technology development. The objective of this research is to develop the right representations and algorithms, implement a prototype, experiment with the appropriate user interfaces to support both initial discovery and the end-user's assessment of the discovered trends, and evaluate the system's ability to enable insightful trends that are not available with normal use of search engines.
Executives, market analysts, science and technology monitors, and government intelligence analysts all are concerned with finding trends. There are established commercial business-intelligence methods for finding and understanding trends in structured data. However, research accomplishments from trend detection in text data have not entered mainstream use because of complicated user experiences and an inability to discern what are truly emerging topics, rather than merely recurring salient topics or linguistic variations on old topics, e.g., "lawsuit" as an old topic and "litigation" as a new topic. If successfully embedded within the familiar search workflow, it is anticipated that trend discovery will be a boon to many knowledge-work professions, leading to early detection of changes that can impact science, technology, government, healthcare, and business performance.
