Clinical decision support (CDS) tools are designed to help healthcare providers make better decisions. Significant and mounting evidence suggests that CDS, when used effectively, can improve health care quality, safety, and effectiveness (2-7). Indeed, a good part of the promise of the electronic health record (EHR) to improve patient care relative to a paper-based healthcare ecosystem rests upon CDS. When CDS works well, healthcare providers can come to depend on it. However, we have identified a number of instances where CDS interventions malfunctioned and either stopped providing correct alerts or began providing incorrect alerts. In many cases, these malfunctions persisted for a long period of time and in some cases, they led to patient harm. We propose a novel anomaly-detection-based method for identifying malfunctions in CDS systems so that they can be corrected. Such approaches have been used in other industries to identify deviations from expected behavior, such as credit card fraud or computer network intrusion detection, but never previously applied to the problem of CDS function and failure. In preliminary work, we have shown that even simple anomaly-detection-based approaches can identify many CDS malfunctions. In the proposed project, we will extend these methods to improve their sensitivity and specificity and validate them at three sites: Brigham and Women's Hospital, The Ohio State University Medical Center and the University of Texas. We will also conduct a qualitative assessment and root cause analyses of CDS malfunctions and develop an open-source modular dashboard and alerting system for tracking them. Our project has three aims: 1) to inventory CDS failures and issues that have occurred in three medical centers and conduct root cause analyses to identify causes, indicators, and potential solutions, 2) to develop and validate generalizable anomaly detection approaches to identifying CDS failures and 3) to create, implement, and test a useful and generalizable CDS dashboard and alert system for real-time monitoring for CDS anomalies.
Clinical decision support systems, such as drug-interaction alerts and preventive care reminders, when used effectively, have been shown to the quality, safety and efficiency of care. However, such systems are complex and sometimes fail - these failures are often not noticed for a long period of time and can lead to patient harm. In the proposed project, we will study the causes of such failures and develop and test anomaly detection systems to detect such failures and alert knowledge engineers about them with the goal of improving the safety and reliability of clinical decision support systems.
