While artificial intelligence (AI) and machine learning (ML) are becoming widely used throughout medicine, the analysis of the cost of an ML model?s predictions has been very limited. For example, an ML model may accurately predict that a trauma patient will have acute traumatic coagulopathy (ATC), a bleeding disorder; however, it may heavily rely on hard-to-measure patient features, like blood pressure or Glasgow Coma Score, to do so. Standard ML techniques do not prioritize timely diagnosis, which is key to minimize death and injury. This idea, which we refer to as cost-aware prediction, is a topic of recent interest in machine learning. However, existing methods have substantial limitations, and their clinical impact has not been demonstrated. This proposal will adopt recent advances in ML and explainable AI to 1) develop improved cost-aware prediction techniques. 2) demonstrate their value using clinical data and 3) integrate them into the electronic medical record. These methods will be applicable in many areas of science and medicine.
Aim 1. Develop a novel feature importance-based approach for cost-aware prediction. No existing approach for cost-aware prediction consistently outperforms the others, and each has its own strengths and weaknesses. This proposal uses recent discoveries in machine learning to design a new algorithm, CoAI, with new strengths and fewer weaknesses. CoAI will substantially improve predictive performance, enable analysis on large datasets, and flexibly work with any ML model. Preliminary results show that CoAI can outperform existing methods. A new public benchmark for cost-aware prediction will be created and used to compare CoAI to existing methods, and CoAI will be published as easy-to-use open-source software.
Aim 2. Evaluate CoAI?s potential for clinical time savings. CoAI?s ability to predict bleeding disorders will be tested on an unprecedentedly detailed dataset that combines trauma hospital data with surveys of doctors and paramedics. Comparing CoAI to the risk scores used in clinical practice will provide explicit estimates of how much time CoAI can save and how many misdiagnoses it can prevent. In preliminary analysis with trauma registry data, CoAI reduces prediction time and increases accuracy relative to an existing risk score.
Aim 3. Incorporate an interactive ML method into the medical record. CoAI will be integrated into the electronic medical record (EMR), using feedback from professional paramedics. Quantitative estimates of time and cost savings and subjective impressions will be gathered from paramedics, and open-ended interviews will be conducted to assess their feelings about interactive machine learning methods like CoAI. These insights will guide future research in interactive machine learning methods, as well as possible clinical work to study CoAI?s impact on decision making in simulated trauma scenarios. Successful completion of this project will allow faster, more accurate diagnosis of acute illness and advance the state of the art in machine learning and artificial intelligence.
Artificial intelligence (AI) is rapidly becoming an integral part of healthcare, from automatic radiology reading and triage to early-warning systems that detect adverse events during surgery. Until recently, AI systems have not been optimized to rely on data that can be gathered quickly or easily. We believe developing AI systems that can intelligently minimize the cost their use imposes on health care providers will lead to more efficient patient risk prediction and diagnosis, and will ultimately save lives.
