Development of a volumetric pancreas segmentation CT dataset for AI applications through trained technologists: a study during the COVID 19 containment phase

Purpose: To evaluate the performance of trained technologists vis-à-vis radiologists for volumetric pancreas segmentation and to assess the impact of supplementary training on their performance. Methods: In this IRB-approved study, 22 technologists were trained in pancreas segmentation on portal venous phase CT through radiologist-led interactive videoconferencing sessions based on an image-rich curriculum. Technologists segmented pancreas in 188 CTs using freehand tools on custom image-viewing software. Subsequent supplementary training included multimedia videos focused on common errors, which were followed by second batch of 159 segmentations. Two radiologists reviewed all cases and corrected inaccurate segmentations. Technologists’ segmentations were compared against radiologists’ segmentations using Dice-Sorenson coefficient (DSC), Jaccard coefficient (JC), and Bland–Altman analysis. Results: Corrections were made in 71 (38%) cases from first batch [26 (37%) oversegmentations and 45 (63%) undersegmentations] and in 77 (48%) cases from second batch [12 (16%) oversegmentations and 65 (84%) undersegmentations]. DSC, JC, false positive (FP), and false negative (FN) [mean (SD)] in first versus second batches were 0.63 (0.15) versus 0.63 (0.16), 0.48 (0.15) versus 0.48 (0.15), 0.29 (0.21) versus 0.21 (0.10), and 0.36 (0.20) versus 0.43 (0.19), respectively. Differences were not significant (p > 0.05). However, range of mean pancreatic volume difference reduced in the second batch [− 2.74 cc (min − 92.96 cc, max 87.47 cc) versus − 23.57 cc (min − 77.32, max 30.19)]. Conclusion: Trained technologists could perform volumetric pancreas segmentation with reasonable accuracy despite its complexity. Supplementary training further reduced range of volume difference in segmentations. Investment into training technologists could augment and accelerate development of body imaging datasets for AI applications.