Rapid and accurate alerting of concerning patient events and conditions remains an important problem in clinical practice. Typical computer-based detection methods developed for this purpose rely on the use of clinical knowledge, such as expert-derived rules, that are incorporated into the monitoring and alerting systems. However, it is often time-consuming and costly to extract and codify such knowledge;hence such systems are typically built to cover only very specific conditions. In addition, it is difficult for an expert to foresee the performance of the deployed systems and their potential drawbacks, especially their false alarm rates. It is not uncommon that computer alerting systems are discarded or must undergo multiple costly modification cycles before they reach clinically acceptable levels of performance. Electronic health record (EHR) repositories today provide an opportunity to test various theories and develop new computational solutions to various clinical problems. The objective of this project is to investigate methods for using the data in such repositories to assist in the development of alerting systems. The project goals include the building of an evidence-driven framework for the evaluation and optimization of alerting systems with the help of past data. The framework will be able to provide early feedback and future performance estimates of an alerting system before it is deployed, which is anticipated to decrease the expert effort required to design such a system and lead to a shorter alerting system design cycle. The evidence-driven framework will be tested and evaluated on multiple clinical conditions and compared to the performance of alerting rules currently deployed at the University of Pittsburgh Medical Center (UPMC). The project investigators consist of a multidisciplinary team with expertise in rule-based alerting in the hospital setting, clinical pharmacy, laboratory medicine, biomedical informatics, statistical machine learning, and knowledge based systems.

Public Health Relevance

There remain numerous opportunities to reduce medical errors by sending computer-based reminders and alerts to clinicians. This project investigates a novel combination of past patient data stored in electronic form and statistical machine-learning methods to help develop and refine computer-based alerts, which are expected to improve healthcare quality and reduce costs.

National Institute of Health (NIH)
National Library of Medicine (NLM)
Research Project (R01)
Project #
Application #
Study Section
Biomedical Library and Informatics Review Committee (BLR)
Program Officer
Sim, Hua-Chuan
Project Start
Project End
Budget Start
Budget End
Support Year
Fiscal Year
Total Cost
Indirect Cost
University of Pittsburgh
Biostatistics & Other Math Sci
Schools of Arts and Sciences
United States
Zip Code
Luo, Zhipeng; Hauskrecht, Milos (2018) Hierarchical Active Learning with Group Proportion Feedback. IJCAI (U S) 2018:2532-2538
Xue, Yanbing; Hauskrecht, Milos (2017) Active Learning of Classification Models with Likert-Scale Feedback. Proc SIAM Int Conf Data Min 2017:28-35
Xue, Yanbing; Hauskrecht, Milos (2017) Efficient Learning of Classification Models from Soft-label Information by Binning and Ranking. Proc Int Fla AI Res Soc Conf 2017:164-169
Luo, Zhipeng; Hauskrecht, Milos (2017) Group-Based Active Learning of Classification Models. Proc Int Fla AI Res Soc Conf 2017:92-97
Batal, Iyad; Cooper, Gregory; Fradkin, Dmitriy et al. (2016) An Efficient Pattern Mining Approach for Event Detection in Multivariate Temporal Data. Knowl Inf Syst 46:115-150
Liu, Zitao; Hauskrecht, Milos (2015) Clinical time series prediction: Toward a hierarchical dynamical system framework. Artif Intell Med 65:5-18
Liu, Zitao; Hauskrecht, Milos (2015) A Regularized Linear Dynamical System Framework for Multivariate Time Series Analysis. Proc Conf AAAI Artif Intell 2015:1798-1804
Naeini, Mahdi Pakdaman; Cooper, Gregory F; Hauskrecht, Milos (2015) Obtaining Well Calibrated Probabilities Using Bayesian Binning. Proc Conf AAAI Artif Intell 2015:2901-2907
Heim, Eric; Hauskrecht, Milos (2015) Sparse Multidimensional Patient Modeling using Auxiliary Confidence Labels. Proceedings (IEEE Int Conf Bioinformatics Biomed) 2015:331-336
Hong, Charmgil; Batal, Iyad; Hauskrecht, Milos (2014) A Mixtures-of-Trees Framework for Multi-Label Classification. Proc ACM Int Conf Inf Knowl Manag 2014:211-220

Showing the most recent 10 out of 30 publications