Rare events occur very infrequently and are thus very difficult to detect. However, when they do occur, their consequences can be quite dramatic quite often in a negative sense. Examples include network intrusions and security breaches, cardiac events, credit card and other types of financial fraud, telecom circuit overloads, and traffic accidents. Timely and accurate detection of rare events is critical. Recent years have seen an explosive growth in the speed of data collection and storage devices. Most organizations collect quantities of data about various processes in their computer systems, at various levels of abstraction, including hardware, operating and communication systems, database query logs, etc. These comprehensive event logs provide a wealth of data, analysis of which has the potential to identify the rare events described earlier. The project includes investigation of the issues in rare class analysis, and development of a suite of techniques to address them. The focus of this project is on supervised learning methods for rare class analysis. Specific tasks include development of novel feature selection schemes and robust predictive models especially suited for rare class problems, and adapting rare class learning algorithms to data streams. The research results will be made publicly available at the project website (www.cs.umn.edu/~kumar/rare.html). The techniques developed, as part of this research will be applicable across a wide spectrum of applications in which one is interested in finding those few, unusual, special cases which are highly significant and of potentially very high value.
