Review of the Bethesda System Atlas does not improve reproducibility or accuracy in the classification of atypical squamous cells of undetermined significance smears

Ann E. Smith, Mark E. Sherman, David R. Scott, Sana O. Tabbara, Lawrence Dworkin, June Olson, John Thompson, Christine Faser, Joy Snell, Mark Schiffman

Research output: Contribution to journalArticle

35 Citations (Scopus)

Abstract

BACKGROUND. The Bethesda System (TBS) and its accompanying atlas were developed to promote uniform diagnosis and reporting of cervical and vaginal cytology, especially with respect to borderline abnormal smears. The authors assessed whether group study of TBS atlas improves the reproducibility and accuracy of the cytopathologic diagnosis of equivocal Papanicolaou smears. METHODS. One hundred 'atypical' smears were divided into pretest and posttest sets containing equal numbers of negative, atypical squamous cells of undetermined significance (ASCUS), and squamous intraepithelial lesion (SIL) diagnoses based on a five-member panel review. Two comparable teams of four pathologists from George Washington University Medical Center (Washington, DC) and Kaiser Permanente (Portland, OR), each comprised of two more experienced cytopathologists and two less experienced pathologists, independently reviewed the 50 pretest slides and classified the slides according to TBS as negative, ASCUS, or SIL. The teams then conducted group study sessions using TBS atlas. After the review, the pathologists independently classified the 50 posttest slides in a similar manner. RESULTS. Pretest, pair-wise interobserver agreement ranged from 30% to 66% compared with 34-62% for posttest agreement. Absolute percent agreement of reviewers' diagnoses with a previously developed consensus diagnosis based on opinions of a five-expert panel (cytopathologic certainty scale) ranged from 44% to 62% for the pretest set and from 40% to 60% for the posttest set. Comparison of the detection of oncogenic human papilloma virus (HPV) DNA by hybrid capture tube test with smears classified as negative, ASCUS, or SIL revealed that seven of eight reviewers did not demonstrate a stronger association between HPV detection and cytologic diagnosis in the posttest set. CONCLUSIONS. Review of TBS atlas by itself does not appear to improve the reproducibility or accuracy of cytologic diagnoses. The lack of improvement was similar among the pathologists involved regardless of experience level or whether they had a close working relation. (C) 2000 American Cancer Society.

Original languageEnglish (US)
Pages (from-to)201-206
Number of pages6
JournalCancer
Volume90
Issue number4
DOIs
StatePublished - Aug 25 2000
Externally publishedYes

Fingerprint

Atlases
Papillomaviridae
Papanicolaou Test
Oncogenic Viruses
Atypical Squamous Cells of the Cervix
Cell Biology
Pathologists
DNA
Squamous Intraepithelial Lesions of the Cervix

Keywords

  • Accuracy
  • Atypical squamous cells of undetermined significance
  • Bethesda System
  • Cervicovaginal
  • Cytopathology
  • Interobserver agreement
  • Papanicolau
  • Reproducibility

ASJC Scopus subject areas

  • Oncology
  • Cancer Research

Cite this

Review of the Bethesda System Atlas does not improve reproducibility or accuracy in the classification of atypical squamous cells of undetermined significance smears. / Smith, Ann E.; Sherman, Mark E.; Scott, David R.; Tabbara, Sana O.; Dworkin, Lawrence; Olson, June; Thompson, John; Faser, Christine; Snell, Joy; Schiffman, Mark.

In: Cancer, Vol. 90, No. 4, 25.08.2000, p. 201-206.

Research output: Contribution to journalArticle

Smith, Ann E. ; Sherman, Mark E. ; Scott, David R. ; Tabbara, Sana O. ; Dworkin, Lawrence ; Olson, June ; Thompson, John ; Faser, Christine ; Snell, Joy ; Schiffman, Mark. / Review of the Bethesda System Atlas does not improve reproducibility or accuracy in the classification of atypical squamous cells of undetermined significance smears. In: Cancer. 2000 ; Vol. 90, No. 4. pp. 201-206.
@article{2c4722c887184632922201ce92e4205e,
title = "Review of the Bethesda System Atlas does not improve reproducibility or accuracy in the classification of atypical squamous cells of undetermined significance smears",
abstract = "BACKGROUND. The Bethesda System (TBS) and its accompanying atlas were developed to promote uniform diagnosis and reporting of cervical and vaginal cytology, especially with respect to borderline abnormal smears. The authors assessed whether group study of TBS atlas improves the reproducibility and accuracy of the cytopathologic diagnosis of equivocal Papanicolaou smears. METHODS. One hundred 'atypical' smears were divided into pretest and posttest sets containing equal numbers of negative, atypical squamous cells of undetermined significance (ASCUS), and squamous intraepithelial lesion (SIL) diagnoses based on a five-member panel review. Two comparable teams of four pathologists from George Washington University Medical Center (Washington, DC) and Kaiser Permanente (Portland, OR), each comprised of two more experienced cytopathologists and two less experienced pathologists, independently reviewed the 50 pretest slides and classified the slides according to TBS as negative, ASCUS, or SIL. The teams then conducted group study sessions using TBS atlas. After the review, the pathologists independently classified the 50 posttest slides in a similar manner. RESULTS. Pretest, pair-wise interobserver agreement ranged from 30{\%} to 66{\%} compared with 34-62{\%} for posttest agreement. Absolute percent agreement of reviewers' diagnoses with a previously developed consensus diagnosis based on opinions of a five-expert panel (cytopathologic certainty scale) ranged from 44{\%} to 62{\%} for the pretest set and from 40{\%} to 60{\%} for the posttest set. Comparison of the detection of oncogenic human papilloma virus (HPV) DNA by hybrid capture tube test with smears classified as negative, ASCUS, or SIL revealed that seven of eight reviewers did not demonstrate a stronger association between HPV detection and cytologic diagnosis in the posttest set. CONCLUSIONS. Review of TBS atlas by itself does not appear to improve the reproducibility or accuracy of cytologic diagnoses. The lack of improvement was similar among the pathologists involved regardless of experience level or whether they had a close working relation. (C) 2000 American Cancer Society.",
keywords = "Accuracy, Atypical squamous cells of undetermined significance, Bethesda System, Cervicovaginal, Cytopathology, Interobserver agreement, Papanicolau, Reproducibility",
author = "Smith, {Ann E.} and Sherman, {Mark E.} and Scott, {David R.} and Tabbara, {Sana O.} and Lawrence Dworkin and June Olson and John Thompson and Christine Faser and Joy Snell and Mark Schiffman",
year = "2000",
month = "8",
day = "25",
doi = "10.1002/1097-0142(20000825)90:4<201::AID-CNCR1>3.0.CO;2-Q",
language = "English (US)",
volume = "90",
pages = "201--206",
journal = "Cancer",
issn = "0008-543X",
publisher = "John Wiley and Sons Inc.",
number = "4",

}

TY - JOUR

T1 - Review of the Bethesda System Atlas does not improve reproducibility or accuracy in the classification of atypical squamous cells of undetermined significance smears

AU - Smith, Ann E.

AU - Sherman, Mark E.

AU - Scott, David R.

AU - Tabbara, Sana O.

AU - Dworkin, Lawrence

AU - Olson, June

AU - Thompson, John

AU - Faser, Christine

AU - Snell, Joy

AU - Schiffman, Mark

PY - 2000/8/25

Y1 - 2000/8/25

N2 - BACKGROUND. The Bethesda System (TBS) and its accompanying atlas were developed to promote uniform diagnosis and reporting of cervical and vaginal cytology, especially with respect to borderline abnormal smears. The authors assessed whether group study of TBS atlas improves the reproducibility and accuracy of the cytopathologic diagnosis of equivocal Papanicolaou smears. METHODS. One hundred 'atypical' smears were divided into pretest and posttest sets containing equal numbers of negative, atypical squamous cells of undetermined significance (ASCUS), and squamous intraepithelial lesion (SIL) diagnoses based on a five-member panel review. Two comparable teams of four pathologists from George Washington University Medical Center (Washington, DC) and Kaiser Permanente (Portland, OR), each comprised of two more experienced cytopathologists and two less experienced pathologists, independently reviewed the 50 pretest slides and classified the slides according to TBS as negative, ASCUS, or SIL. The teams then conducted group study sessions using TBS atlas. After the review, the pathologists independently classified the 50 posttest slides in a similar manner. RESULTS. Pretest, pair-wise interobserver agreement ranged from 30% to 66% compared with 34-62% for posttest agreement. Absolute percent agreement of reviewers' diagnoses with a previously developed consensus diagnosis based on opinions of a five-expert panel (cytopathologic certainty scale) ranged from 44% to 62% for the pretest set and from 40% to 60% for the posttest set. Comparison of the detection of oncogenic human papilloma virus (HPV) DNA by hybrid capture tube test with smears classified as negative, ASCUS, or SIL revealed that seven of eight reviewers did not demonstrate a stronger association between HPV detection and cytologic diagnosis in the posttest set. CONCLUSIONS. Review of TBS atlas by itself does not appear to improve the reproducibility or accuracy of cytologic diagnoses. The lack of improvement was similar among the pathologists involved regardless of experience level or whether they had a close working relation. (C) 2000 American Cancer Society.

AB - BACKGROUND. The Bethesda System (TBS) and its accompanying atlas were developed to promote uniform diagnosis and reporting of cervical and vaginal cytology, especially with respect to borderline abnormal smears. The authors assessed whether group study of TBS atlas improves the reproducibility and accuracy of the cytopathologic diagnosis of equivocal Papanicolaou smears. METHODS. One hundred 'atypical' smears were divided into pretest and posttest sets containing equal numbers of negative, atypical squamous cells of undetermined significance (ASCUS), and squamous intraepithelial lesion (SIL) diagnoses based on a five-member panel review. Two comparable teams of four pathologists from George Washington University Medical Center (Washington, DC) and Kaiser Permanente (Portland, OR), each comprised of two more experienced cytopathologists and two less experienced pathologists, independently reviewed the 50 pretest slides and classified the slides according to TBS as negative, ASCUS, or SIL. The teams then conducted group study sessions using TBS atlas. After the review, the pathologists independently classified the 50 posttest slides in a similar manner. RESULTS. Pretest, pair-wise interobserver agreement ranged from 30% to 66% compared with 34-62% for posttest agreement. Absolute percent agreement of reviewers' diagnoses with a previously developed consensus diagnosis based on opinions of a five-expert panel (cytopathologic certainty scale) ranged from 44% to 62% for the pretest set and from 40% to 60% for the posttest set. Comparison of the detection of oncogenic human papilloma virus (HPV) DNA by hybrid capture tube test with smears classified as negative, ASCUS, or SIL revealed that seven of eight reviewers did not demonstrate a stronger association between HPV detection and cytologic diagnosis in the posttest set. CONCLUSIONS. Review of TBS atlas by itself does not appear to improve the reproducibility or accuracy of cytologic diagnoses. The lack of improvement was similar among the pathologists involved regardless of experience level or whether they had a close working relation. (C) 2000 American Cancer Society.

KW - Accuracy

KW - Atypical squamous cells of undetermined significance

KW - Bethesda System

KW - Cervicovaginal

KW - Cytopathology

KW - Interobserver agreement

KW - Papanicolau

KW - Reproducibility

UR - http://www.scopus.com/inward/record.url?scp=0034714217&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0034714217&partnerID=8YFLogxK

U2 - 10.1002/1097-0142(20000825)90:4<201::AID-CNCR1>3.0.CO;2-Q

DO - 10.1002/1097-0142(20000825)90:4<201::AID-CNCR1>3.0.CO;2-Q

M3 - Article

C2 - 10966559

AN - SCOPUS:0034714217

VL - 90

SP - 201

EP - 206

JO - Cancer

JF - Cancer

SN - 0008-543X

IS - 4

ER -