A French retrospective study evaluated CINA-PE (Avicenna.AI), an AI tool for pulmonary embolism (PE) detection, using 196 CT pulmonary angiograms (15.8% PE-positive). Radiology residents initially read the CTPAs without AI, and then, after a 2-month interval, with AI assistance, to determine the presence, absence, or indeterminacy of PE. The assistance of CINA-PE led to increased diagnostic sensitivity (from 81.7% to 92.5%) and specificity (from 97.8% to 99.2%). Additionally, the AI support significantly enhanced reader agreement, elevating the kappa value from 0.77 to 0.88.
Read full study.
Abstract
Purpose
To compare radiology residents’ diagnostic performances to detect pulmonary emboli (PEs) on CT pulmonary angiographies (CTPAs) with deep-learning (DL)–based algorithm support and without.
Methods
Fully anonymized CTPAs (n = 207) of patients suspected of having acute PE served as input for PE detection using a previously trained and validated DL-based algorithm. Three residents in their first three years of training, blinded to the index report and clinical history, read the CTPAs first without, and 2 months later with the help of artificial intelligence (AI) output, to diagnose PE as present, absent or indeterminate. We evaluated concordances and discordances with the consensus-reading results of two experts in chest imaging.
Results
Because the AI algorithm failed to analyze 11 CTPAs, 196 CTPAs were analyzed; 31 (15.8 %) were PE-positive. Good-classification performance was higher for residents with AI-algorithm support than without (AUROCs: 0.958 [95 % CI: 0.921–0.979] vs. 0.894 [95 % CI: 0.850–0.931], p < 0.001, respectively). The main finding was the increased sensitivity of residents’ diagnoses using the AI algorithm (92.5 % vs. 81.7 %, respectively). Concordance between residents (kappa: 0.77 [95 % CI: 0.76–0.78]; p < 0.001) improved with AI-algorithm use (kappa: 0.88 [95 % CI: 0.87–0.89]; p < 0.001).
Conclusion
The AI algorithm we used improved between-resident agreements to interpret CTPAs for suspected PE and, hence, their diagnostic performances.