A German retrospective study evaluated the efficacy of mdbrain (mediaire), an AI-based tool, in identifying cerebral aneurysms using time-of-flight magnetic resonance angiography (TOF-MRA) across 186 participants, covering 54 aneurysms. The research protocol included an initial assessment with AI assistance, followed by a six-week washout period, and a subsequent re-assessment without the AI tool, including six readers with diverse experience levels. When evaluating the group of readers as a whole, the results showed that mdbrain significantly improved diagnostic accuracy, increasing specificity from 93.3% to 95.4% and reducing false positives per case from 0.09 to 0.06. Sensitivity for detecting lesions rose from 70.1% to 73.5%, and patient-level sensitivity from 66.3% to 71.2%. However, no statistically significant differences were found in the performance of individual readers. Additionally, while the study noted a reduction in reading times for four readers using the software, two readers experienced increased times. This suggests that further testing of the tool in clinical settings is necessary to understand its varying impacts on efficiency.
Read full study
Impact of an AI software on the diagnostic performance and reading time for the detection of cerebral aneurysms on time of flight MR-angiography
Neuroradiology, 2024
Abstract
Purpose: To evaluate the impact of an AI-based software trained to detect cerebral aneurysms on TOF-MRA on the diagnostic performance and reading times across readers with varying experience levels.
Methods: One hundred eighty-six MRI studies were reviewed by six readers to detect cerebral aneurysms. Initially, readings were assisted by the CNN-based software mdbrain. After 6 weeks, a second reading was conducted without software assistance. The results were compared to the consensus reading of two neuroradiological specialists and sensitivity (lesion and patient level), specificity (patient level), and false positives per case were calculated for the group of all readers, for the subgroup of physicians, and for each individual reader. Also, reading times for each reader were measured.
Results: The dataset contained 54 aneurysms. The readers had no experience (three medical students), 2 years experience (resident in neuroradiology), 6 years experience (radiologist), and 12 years (neuroradiologist). Significant improvements of overall specificity and the overall number of false positives per case were observed in the reading with AI support. For the physicians, we found significant improvements of sensitivity on lesion and patient level and false positives per case. Four readers experienced reduced reading times with the software, while two encountered increased times.
Conclusion: In the reading with the AI-based software, we observed significant improvements in terms of specificity and false positives per case for the group of all readers and significant improvements of sensitivity and false positives per case for the physicians. Further studies are needed to investigate the effects of the AI-based software in a prospective setting.