Interobserver agreement and the impact of mentorship on the diagnosis of inflammatory bowel disease–associated dysplasia among subspecialist gastrointestinal pathologists

Lindsay Alpert, Namrata Setia, Huaibin Mabel Ko, Stephen M. Lagana, Meredith E. Pittman, Melanie Johncilla, Michael G. Drage, Lei Zhao, Marcela A. Salomao, Xiaoyan Liao, Won Tak Choi, Sarah M. Jenkins, John Hart, Noam Harpaz, Lysandra Voltaggio, Gregory Y. Lauwers, Robert Odze, Helen Remotti, Thomas C. Smyrk, Rondell P. Graham

Research output: Contribution to journalArticlepeer-review

Abstract

The diagnosis of inflammatory bowel disease (IBD)–associated dysplasia is challenging, and past studies have demonstrated considerable interobserver variability in such diagnoses. This study aimed to assess interobserver agreement in IBD dysplasia diagnoses among subspecialty GI pathologists and to explore the impact of mentorship on diagnostic variability. Twelve GI pathologist mentees and 7 GI pathologist mentors reviewed 163 digitized slides. Participants rendered a diagnosis of negative for dysplasia, indefinite for dysplasia, low-grade dysplasia, or high-grade dysplasia and provided a confidence level for each case. Interobserver agreement and reliability were assessed using Cohen’s and Fleiss’ kappa (κ) statistics and intraclass correlation coefficient (ICC) analysis. The overall κ coefficient was 0.42 (95% CI: 0.38–0.46). The overall ICC was 0.67 (95% CI: 0.62–0.72). Κ coefficients ranged from 0.31 to 0.49 for mentor/mentee pairs and from 0.34 to 0.55 for pairs of mentees of the same mentor. The combined κ coefficient was 0.44 (95% CI: 0.39–0.48) for all mentees and 0.39 (95% CI: 0.34–0.43) for all mentors. Common features in low agreement cases included mucosal atrophy, areas of stark contrast, serrations, decreased goblet cells, absent surface epithelium, and poor orientation. Participants were confident in most diagnoses, and increased confidence levels generally correlated with higher interobserver agreement. Interobserver agreement among subspecialist GI pathologists in this curated cohort of IBD dysplasia cases was fair to moderate. Mentorship during GI pathology fellowship does not appear to be a significant factor contributing to interobserver variability, but increased experience also does not seem to improve interobserver agreement.

Original languageEnglish (US)
Pages (from-to)1061-1069
Number of pages9
JournalVirchows Archiv
Volume478
Issue number6
DOIs
StatePublished - Jun 2021

Keywords

  • Dysplasia
  • Inflammatory bowel disease
  • Interobserver variability
  • Mentorship

ASJC Scopus subject areas

  • Pathology and Forensic Medicine
  • Molecular Biology
  • Cell Biology

Fingerprint

Dive into the research topics of 'Interobserver agreement and the impact of mentorship on the diagnosis of inflammatory bowel disease–associated dysplasia among subspecialist gastrointestinal pathologists'. Together they form a unique fingerprint.

Cite this