Automated CT angiography collateral scoring in anterior large vessel occlusion stroke: A multireader study

Background: Computed tomography (CT) angiography collateral score (CTA-CS) is an important clinical outcome predictor following mechanical thrombectomy for ischemic stroke with large vessel occlusion (LVO). The present multireader study aimed to evaluate the performance of e-CTA software for automated assistance in CTA-CS scoring. Materials and Methods: Brain CTA images of 56 patients with anterior LVO were retrospectively processed. Twelve readers of various clinical training, including junior neuroradiologists, senior neuroradiologists, and neurologists graded collateral flow using visual CTA-CS scale in two sessions separated by a washout period. Reference standard was the consensus of three expert readers. Duration of reading time, inter-rater reliability, and statistical comparison of readers’ performance metrics were analyzed between the e-CTA assisted and unassisted sessions. Results: e-CTA assistance resulted in significant increase in mean accuracy (58.6% to 67.5%, p = 0.003), mean F1 score (0.574 to 0.676, p = 0.002), mean precision (58.8% to 68%, p = 0.007), and mean recall (58.7% to 69.9%, p = 0.002), especially with slight filling deficit (CTA-CS 2 and 3). Mean reading time was reduced across all readers (103.4 to 59.7 s, p = 0.001), and inter-rater agreement in CTA-CS assessment was increased (Krippendorff's alpha 0.366 to 0.676). Optimized occlusion laterality detection was also noted with mean accuracy (92.9% to 96.8%, p = 0.009). Conclusion: Automated assistance for CTA-CS using e-CTA software provided helpful decision support for readers in terms of improving scoring accuracy and reading efficiency for physicians with a range of experience and training backgrounds and leading to significant improvements in inter-rater agreement.