An artificial intelligence tool proposed to support clinicians’ decision-making regarding hospitalized patients at risk of sepsis has the unusual capability of considering uncertainty and suggesting the demographic data, vital signs, and laboratory test results required to improve predictive performance.
The system, called “SepsisLab,” was developed based on feedback from doctors and nurses who treat patients in emergency departments and intensive care units, where sepsis, the body’s overreaction to infection, is most prevalent. They expressed dissatisfaction with existing AI-assisted tools that generate patient risk prediction scores using only electronic medical records, without input data from clinicians.
Scientists at Ohio State University designed SepsisLab to predict a patient’s sepsis risk within four hours. But over time, the system identifies missing patient information, quantifies its importance, and visually shows clinicians how specific information affects the final risk prediction. Experiments using a combination of publicly available and proprietary patient data showed that adding 8% of the recommended data improved the system’s sepsis prediction accuracy by 11%.
“Existing models represent a more traditional human-AI competition paradigm, which generates a lot of nuisance false alarms in ICUs and emergency rooms without consulting clinicians,” said Ping Zhang, an associate professor of computer science and engineering and biomedical informatics at The Ohio State University and lead author of the study.
“Our idea is that by adopting the concept of ‘AI in the human loop,’ we need to involve AI at every intermediate step in the decision-making. We’re not just developing the tools, we’ve also brought doctors into the project. It’s a true collaboration between computer scientists and clinicians to develop a human-centric system where the doctor is in charge.”
The study was published August 24th. KDD ’24: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. The research will be presented orally at SIGKDD 2024 in Barcelona, Spain on Wednesday (August 28).
Sepsis is a life-threatening medical emergency that can rapidly lead to organ failure, but it is difficult to diagnose because symptoms such as fever, low blood pressure, rapid heart rate and difficulty breathing can mimic many other medical conditions. The study builds on a machine learning model previously developed by Chan and his colleagues to estimate the optimal time to administer antibiotics to patients suspected of having sepsis.
SepsisLab is designed to make risk predictions quickly, but generates new predictions every hour as new patient data is added to the system.
“When a patient first comes to the hospital, there are a lot of values missing, especially in clinical tests,” said Changchang Yin, a computer science and engineering doctoral student in Zhang’s medical artificial intelligence lab and lead author of the paper.
In most AI models, missing data points are accounted for with a single assigned value (a process known as imputation), “but imputation models can suffer from uncertainty that can propagate to downstream predictive models,” Yin said.
“If the imputation model cannot accurately impute a missing value, and it is a very important value, then that variable should be observed. Our active sensing algorithm aims to find such missing values and tell the clinician what additional variables they may need to observe – variables that can increase the accuracy of the predictive model.”
Equally important to removing uncertainty from the system over time is providing actionable recommendations to clinicians, including laboratory tests ranked based on their value to the diagnostic process and estimates of how a patient’s sepsis risk changes in response to specific clinical treatments.
Experiments showed that adding 8% of new data from lab tests, vital signs, and other high-value variables reduced the uncertainty propagated into the model by 70% and improved the accuracy of sepsis risk by 11%.
“The algorithm can select the most important variables, and the physician’s actions reduce the uncertainty,” said Chan, who is also a principal faculty member at The Ohio State University’s Translational Data Analytics Institute. “This fundamental mathematical research is the most important innovation and the backbone of our research.”
Zhang believes human-centric AI will be part of the future of healthcare, but only if it interacts with clinicians in a way that encourages them to trust the system.
“This isn’t about building an AI system that can take over the world,” he said. “Healthcare is at its heart about hypothesis testing and making decisions every minute that aren’t simply ‘yes’ or ‘no.’ We envision using AI to help the human at the center of that interaction feel like a superhuman.”
The research was supported by the National Science Foundation, the National Institutes of Health, and the Ohio State University President’s Research Excellence Promotion Grant. Zhang has received additional funding from the NIH to continue collaborating with clinicians on this research.
Additional co-authors include Jeffrey Caterino of The Ohio State University Wexner Medical Center, Bingsheng Yao and Dacuo Wang of Northeastern University, and Ping-Yu Chen of IBM Research.
sauce:
Journal References:
Yoon C., etc. (2024) SepsisLab: Early sepsis prediction through uncertainty quantification and active sensing. KDD ’24: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining.. doi.org/10.1145/3637528.3671586.
