AI Tool Predicts Blood Clot Risk in Hospitalized Pediatric Patients
Vanderbilt researchers have found that clinicians were reluctant to follow the treatment recommendations of a blood clot risk prediction model despite its accuracy.
Source: Getty Images
By - Researchers at Vanderbilt University Medical Center (VUMC) have developed an artificial intelligence (AI) tool capable of accurately identifying the risk of blood clots in pediatric patients, but found that clinicians often hesitated to follow the tool’s recommendations.
Blood clots are rare in pediatric patients, but patients who do develop them can face longer hospital stays and an increased risk of adverse outcomes as a result, the research team indicated.
To address this, the researchers set out to develop and validate an algorithm to identify high-risk patients.
They began by assessing the electronic medical records (EMRs) of 110,000 patients from the Monroe Carell Jr. Children’s Hospital at Vanderbilt. From there, the team flagged 11 factors associated with blood clot risk, including whether a patient had undergone surgery, had infectious disease or cardiology consults, or received certain diagnoses.
Using this information, the researchers developed a predictive model to automatically assess EMRs and calculate a risk score for each pediatric hospital admission on a daily basis. Doing so allowed the research team to better focus on the patients at highest risk.
The risk prediction tool was utilized over the course of 15 months during the Children’s Likelihood of Thrombosis (CLOT) clinical trial, which included 17,000 patients.
The patient cohort was divided into a study group and a control, and risk scores were generated by the AI tool for participants in both groups. However, the risk scores for the study group were shared with their care teams while those for the control group were not.
For patients in the study group, risk scores were provided alongside recommendations to initiate anti-thrombolytic therapy, which helps prevent the development of blood clots. Those in the control group were flagged as high risk by their treating clinicians and also received therapy to prevent blood clots.
In those patients receiving blood thinners as part of their anti-thrombolytic therapy, no bleeding complications were observed across either group. Further, no significant difference in blood clot rates was observed among the two groups at the end of the trial.
However, the researchers also found that recommendations to initiate anti-thrombolytic therapy in study group patients were only followed by clinicians less than 26 percent of the time. These clinicians expressed concern that following the recommendation could lead to a major bleed, though this outcome was not observed in the study.
The researchers explained that these insights could be used to inform the deployment of AI in healthcare.
“There is going to be more and more AI in healthcare. Having a system established where we can assess these (models) will allow us to provide safer and more effective care to our patients,” said Shannon Walker, MD, assistant professor of Pathology, Microbiology and Immunology and Pediatrics, and the paper’s first author, in the press release.
“This study demonstrates that a pragmatic patient-level, randomized, controlled trial is the most ethical and effective way to assess whether AI tools are safe and effective,” noted co-author Daniel Byrne, MS, director of AI Research at the Advanced Vanderbilt Artificial Intelligence Laboratory (AVAIL) and the Department of Biostatistics.
The research team further underscored that the clinical trial was necessary to identify why the implementation of the model was unsuccessful, noting that this was likely due to reluctance to accept the tool’s recommendations, rather than a failure of the model itself.
To further study the feasibility of assessing the value of predictive AI models in healthcare, the researchers are planning to undertake another clinical trial looking at clinicians’ reluctance to follow the AI’s recommendations and how to overcome those barriers.
“We need to make sure these models are performing as expected,” Walker said. “The infrastructure from this trial will allow for large study populations, to determine whether interventions that use artificial intelligence are safe and effective, and to help identify the patients who may benefit the most.”
