A Swiss single-center study evaluated Aidoc’s incidental pulmonary embolism (iPE) detection tool on chest CT scans conducted for non-PE reasons. The study included 5,298 patients across two cohorts: 1,964 patients scanned pre-AI implementation (cohort 1) and 3,334 post-AI implementation (cohort 2).
In cohort 1, 42 iPEs were detected, with radiologists missing 50% of these cases. After AI integration in cohort 2, the radiologists’ miss rates dropped to 7.1%, with specificity remained high at 99% across both groups. The AI integration led to a significant improvement in detection rates (p < 0.001), especially enhancing identification of iPEs in the right lower lobe. Standalone, Aidoc’s tool maintained a sensitivity of 95% in cohort 1 and 90% in cohort 2, with PPV improving from 69% to 95%, while specificity and NPV remained at 99% across both cohorts.
Despite improved diagnostic accuracy, 90-day mortality rates did not differ significantly between cohorts (hazard ratio 0.95; p = 0.88). These findings suggest that while Aidoc’s AI is a valuable diagnostic aid, further research is needed to assess its impact on long-term patient outcomes and management strategies.
Read full study
Implementation of an AI algorithm in clinical practice to reduce missed incidental pulmonary embolisms on chest CT and its impact on short-term survival
Investigative Radiology, 2024
Abstract
Objectives
A substantial number of incidental pulmonary embolisms (iPEs) in computed tomography scans are missed by radiologists in their daily routine. This study analyzes the radiological reports of iPE cases before and after implementation of an artificial intelligence (AI) algorithm for iPE detection. Furthermore, we investigate the anatomic distribution patterns within missed iPE cases and mortality within a 90-day follow-up in patients before and after AI use.
Materials and methods
This institutional review board-approved observational single-center study included 5298 chest computed tomography scans performed for reasons other than suspected pulmonary embolism (PE). We compared 2 cohorts: cohort 1, consisting of 1964 patients whose original radiology reports were generated before the implementation of an AI algorithm, and cohort 2, consisting of 3334 patients whose scans were analyzed after the implementation of an Food and Drug Administration-approved and CE-certified AI algorithm for iPE detection (Aidoc Medical, Tel Aviv, Israel). For both cohorts, any discrepancies between the original radiology reports and the AI results were reviewed by 2 thoracic imaging subspecialized radiologists. In the original radiology report and in case of discrepancies with the AI algorithm, the expert review served as reference standard. Sensitivity, specificity, prevalence, negative predictive value (NPV), and positive predictive value (PPV) were calculated. The rates of missed iPEs in both cohorts were compared statistically using STATA (Version 17.1). Kaplan-Meier curves and Cox proportional hazards models were used for survival analysis.
Results
In cohort 1 (mean age 70.6 years, 48% female [n = 944], 52% male [n = 1020]), the prevalence of confirmed iPE was 2.2% (n = 42), and the AI detected 61 suspicious iPEs, resulting in a sensitivity of 95%, a specificity of 99%, a PPV of 69%, and an NPV of 99%. Radiologists missed 50% of iPE cases in cohort 1. In cohort 2 (mean age 69 years, 47% female [n = 1567], 53% male [n = 1767]), the prevalence of confirmed iPEs was 1.7% (56/3334), with AI detecting 59 suspicious cases (sensitivity 90%, specificity 99%, PPV 95%, NPV 99%). The rate of missed iPEs by radiologists dropped to 7.1% after AI implementation, showing a significant improvement (P < 0.001). Most overlooked iPEs (61%) were in the right lower lobe. The survival analysis showed no significantly decreased 90-day mortality rate, with a hazards ratio of 0.95 (95% confidence interval, 0.45-1.96; P = 0.88).
Conclusions
The implementation of an AI algorithm significantly reduced the rate of missed iPEs from 50% to 7.1%, thereby enhancing diagnostic accuracy. Despite this improvement, the 90-day mortality rate remained unchanged. These findings highlight the AI tool's potential to assist radiologists in accurately identifying iPEs, although its implementation does not significantly affect short-term survival. Notably, most missed iPEs were located in the right lower lobe, suggesting that radiologists should pay particular attention to this area during evaluations.