The Bethesda Interobserver Reproducibility Study (BIRST): A web-based assessment of the Bethesda 2001 system for classifying cervical cytology

Mark E. Sherman, Abhijit Dasgupta, Mark Schiffman, Ritu Nayar, Diane Solomon

Research output: Contribution to journalArticle

51 Citations (Scopus)

Abstract

BACKGROUND. The Bethesda System (TBS) along with its companion atlas was updated in 2001 to improve standardization, clarity, and reproducibility of cervical cytology reporting. METHODS. The authors used a novel web-based format to compare assessments of 77 images demonstrating a range of classical and borderline cytologic changes by a self-selected group of United States cytotechnologists (n = 216) and pathologists (n = 185). RESULTS. Participants were highly experienced, with 71.2% of cytotechnologists and 53.0% of pathologists reporting >10 years of practice. The mean percentage of exact agreement with the panel was slightly though significantly higher for cytotechnologists (57.0%) compared with pathologists (53.4%), adjusted for experience (P = .004); cervical cytology percentage effort (P = .0005); or cervical accession volume (P = .0002). Compared with the TBS panel, exact agreement was achieved for 55.1% of image ratings compared with 82.3% agreement at the level of Negative vs non-Negative for images with a single-panel interpretation. Agreement with the panel was highest for images classified as Low-Grade Squamous Intraepithelial Lesion and lowest for Atypical Squamous Cells qualified as either of Undetermined Significance or Cannot Exclude a High-Grade Squamous Intraepithelial Lesion. Reviewers were less sensitive in identifying high-grade glandular lesions than they were in identifying high-grade squamous lesions at any threshold (P < .001). CONCLUSIONS. Morphologic appearances of images were more important determinants than participants' academic or professional degrees with regard to interobserver reproducibility in classifying cervical cytology images. Experienced cytotechnologists and pathologists performed similarly. Participants achieved higher sensitivity for identifying high-grade squamous lesions than they did for high-grade glandular lesions. These findings demonstrated that web-based studies may be useful in assessing interobserver agreement in classifying images.

Original languageEnglish (US)
Pages (from-to)15-25
Number of pages11
JournalCancer
Volume111
Issue number1
DOIs
StatePublished - Feb 25 2007

Fingerprint

Cell Biology
Atlases
Pathologists
Squamous Intraepithelial Lesions of the Cervix

Keywords

  • Bethesda System
  • Cervix
  • Cytology
  • Internet
  • Reproducibility
  • Screening

ASJC Scopus subject areas

  • Cancer Research
  • Oncology

Cite this

The Bethesda Interobserver Reproducibility Study (BIRST) : A web-based assessment of the Bethesda 2001 system for classifying cervical cytology. / Sherman, Mark E.; Dasgupta, Abhijit; Schiffman, Mark; Nayar, Ritu; Solomon, Diane.

In: Cancer, Vol. 111, No. 1, 25.02.2007, p. 15-25.

Research output: Contribution to journalArticle

Sherman, Mark E. ; Dasgupta, Abhijit ; Schiffman, Mark ; Nayar, Ritu ; Solomon, Diane. / The Bethesda Interobserver Reproducibility Study (BIRST) : A web-based assessment of the Bethesda 2001 system for classifying cervical cytology. In: Cancer. 2007 ; Vol. 111, No. 1. pp. 15-25.
@article{388ae160972648a78efd282eadc2134c,
title = "The Bethesda Interobserver Reproducibility Study (BIRST): A web-based assessment of the Bethesda 2001 system for classifying cervical cytology",
abstract = "BACKGROUND. The Bethesda System (TBS) along with its companion atlas was updated in 2001 to improve standardization, clarity, and reproducibility of cervical cytology reporting. METHODS. The authors used a novel web-based format to compare assessments of 77 images demonstrating a range of classical and borderline cytologic changes by a self-selected group of United States cytotechnologists (n = 216) and pathologists (n = 185). RESULTS. Participants were highly experienced, with 71.2{\%} of cytotechnologists and 53.0{\%} of pathologists reporting >10 years of practice. The mean percentage of exact agreement with the panel was slightly though significantly higher for cytotechnologists (57.0{\%}) compared with pathologists (53.4{\%}), adjusted for experience (P = .004); cervical cytology percentage effort (P = .0005); or cervical accession volume (P = .0002). Compared with the TBS panel, exact agreement was achieved for 55.1{\%} of image ratings compared with 82.3{\%} agreement at the level of Negative vs non-Negative for images with a single-panel interpretation. Agreement with the panel was highest for images classified as Low-Grade Squamous Intraepithelial Lesion and lowest for Atypical Squamous Cells qualified as either of Undetermined Significance or Cannot Exclude a High-Grade Squamous Intraepithelial Lesion. Reviewers were less sensitive in identifying high-grade glandular lesions than they were in identifying high-grade squamous lesions at any threshold (P < .001). CONCLUSIONS. Morphologic appearances of images were more important determinants than participants' academic or professional degrees with regard to interobserver reproducibility in classifying cervical cytology images. Experienced cytotechnologists and pathologists performed similarly. Participants achieved higher sensitivity for identifying high-grade squamous lesions than they did for high-grade glandular lesions. These findings demonstrated that web-based studies may be useful in assessing interobserver agreement in classifying images.",
keywords = "Bethesda System, Cervix, Cytology, Internet, Reproducibility, Screening",
author = "Sherman, {Mark E.} and Abhijit Dasgupta and Mark Schiffman and Ritu Nayar and Diane Solomon",
year = "2007",
month = "2",
day = "25",
doi = "10.1002/cncr.22423",
language = "English (US)",
volume = "111",
pages = "15--25",
journal = "Cancer",
issn = "0008-543X",
publisher = "John Wiley and Sons Inc.",
number = "1",

}

TY - JOUR

T1 - The Bethesda Interobserver Reproducibility Study (BIRST)

T2 - A web-based assessment of the Bethesda 2001 system for classifying cervical cytology

AU - Sherman, Mark E.

AU - Dasgupta, Abhijit

AU - Schiffman, Mark

AU - Nayar, Ritu

AU - Solomon, Diane

PY - 2007/2/25

Y1 - 2007/2/25

N2 - BACKGROUND. The Bethesda System (TBS) along with its companion atlas was updated in 2001 to improve standardization, clarity, and reproducibility of cervical cytology reporting. METHODS. The authors used a novel web-based format to compare assessments of 77 images demonstrating a range of classical and borderline cytologic changes by a self-selected group of United States cytotechnologists (n = 216) and pathologists (n = 185). RESULTS. Participants were highly experienced, with 71.2% of cytotechnologists and 53.0% of pathologists reporting >10 years of practice. The mean percentage of exact agreement with the panel was slightly though significantly higher for cytotechnologists (57.0%) compared with pathologists (53.4%), adjusted for experience (P = .004); cervical cytology percentage effort (P = .0005); or cervical accession volume (P = .0002). Compared with the TBS panel, exact agreement was achieved for 55.1% of image ratings compared with 82.3% agreement at the level of Negative vs non-Negative for images with a single-panel interpretation. Agreement with the panel was highest for images classified as Low-Grade Squamous Intraepithelial Lesion and lowest for Atypical Squamous Cells qualified as either of Undetermined Significance or Cannot Exclude a High-Grade Squamous Intraepithelial Lesion. Reviewers were less sensitive in identifying high-grade glandular lesions than they were in identifying high-grade squamous lesions at any threshold (P < .001). CONCLUSIONS. Morphologic appearances of images were more important determinants than participants' academic or professional degrees with regard to interobserver reproducibility in classifying cervical cytology images. Experienced cytotechnologists and pathologists performed similarly. Participants achieved higher sensitivity for identifying high-grade squamous lesions than they did for high-grade glandular lesions. These findings demonstrated that web-based studies may be useful in assessing interobserver agreement in classifying images.

AB - BACKGROUND. The Bethesda System (TBS) along with its companion atlas was updated in 2001 to improve standardization, clarity, and reproducibility of cervical cytology reporting. METHODS. The authors used a novel web-based format to compare assessments of 77 images demonstrating a range of classical and borderline cytologic changes by a self-selected group of United States cytotechnologists (n = 216) and pathologists (n = 185). RESULTS. Participants were highly experienced, with 71.2% of cytotechnologists and 53.0% of pathologists reporting >10 years of practice. The mean percentage of exact agreement with the panel was slightly though significantly higher for cytotechnologists (57.0%) compared with pathologists (53.4%), adjusted for experience (P = .004); cervical cytology percentage effort (P = .0005); or cervical accession volume (P = .0002). Compared with the TBS panel, exact agreement was achieved for 55.1% of image ratings compared with 82.3% agreement at the level of Negative vs non-Negative for images with a single-panel interpretation. Agreement with the panel was highest for images classified as Low-Grade Squamous Intraepithelial Lesion and lowest for Atypical Squamous Cells qualified as either of Undetermined Significance or Cannot Exclude a High-Grade Squamous Intraepithelial Lesion. Reviewers were less sensitive in identifying high-grade glandular lesions than they were in identifying high-grade squamous lesions at any threshold (P < .001). CONCLUSIONS. Morphologic appearances of images were more important determinants than participants' academic or professional degrees with regard to interobserver reproducibility in classifying cervical cytology images. Experienced cytotechnologists and pathologists performed similarly. Participants achieved higher sensitivity for identifying high-grade squamous lesions than they did for high-grade glandular lesions. These findings demonstrated that web-based studies may be useful in assessing interobserver agreement in classifying images.

KW - Bethesda System

KW - Cervix

KW - Cytology

KW - Internet

KW - Reproducibility

KW - Screening

UR - http://www.scopus.com/inward/record.url?scp=33847111559&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33847111559&partnerID=8YFLogxK

U2 - 10.1002/cncr.22423

DO - 10.1002/cncr.22423

M3 - Article

C2 - 17186503

AN - SCOPUS:33847111559

VL - 111

SP - 15

EP - 25

JO - Cancer

JF - Cancer

SN - 0008-543X

IS - 1

ER -