Integrating eye-tracking data that maps how radiologists interpret X-rays into deep learning AI algorithms could be key to developing more “human-centered” AI, according to a recent study. There is sex.
A group led by researchers in Lisbon, Portugal, conducted a systematic literature review on the use of gaze-driven data in chest X-ray deep learning (DL) models and suggested improvements that could ultimately enhance the approach. .
“There is a clear need for more human-centric technologies to bridge the gap between DL systems and human understanding,” said first author José Neves of the University of Lisbon and colleagues. The study was published in the March issue of the magazine. European Journal of Radiology.
Although DL models have shown remarkable proficiency in a variety of radiology tasks, the internal decision-making processes remain largely opaque, the authors explained. This creates a so-called “black box” problem, they write, where the logic that leads to a model’s decisions is not accessible to human scrutiny.
A promising way to overcome this problem is to integrate data from studies using eye-tracking hardware and software to characterize how radiologists read normal and abnormal chest radiographs. , the authors added.
This data focuses, for example, on saccades (rapid eye movements that occur when an observer shifts their gaze from one point of interest to another) and fixations (periods in which the eyes are relatively still). and can be presented as an attention map, the researchers said. I have written.
Differences in saccades, fixations, and attention heatmaps in an eye-tracking study of radiologists. Image provided by: European Journal of Radiology.
To explore the best way to integrate such data, the group conducted a systematic review to take a closer look at current methods. The initial search yielded 180 research papers and ultimately included 60 of his papers for detailed analysis.
The researchers primarily answered three questions:
- What architectures and fusion techniques are available to integrate eye-tracking data into deep learning approaches to localize and predict lesions?
- How is eye tracking data preprocessed before being incorporated into a multimodal deep learning architecture?
- How can gaze data facilitate explainability in multimodal deep learning architectures?
Ultimately, the researchers say, incorporating gaze data ensures that the features selected by the DL model match the image features that radiologists believe are relevant to the diagnosis. They suggested that these models could therefore be easier to interpret and their decision-making processes more transparent.
They noted that the importance of this review is that it provides concrete answers regarding the role of eye movement data and how to best integrate it into radiology DL algorithms.
“According to our understanding, our study conducts a comprehensive literature review and comparative analysis of features present in gaze data that may lead to deep learning models that are more suitable to support clinical practice.” “This is the first study of its kind,” the researchers concluded.
You can read the full article here.
