Interobserver agreement among endosonographers for endoscopic ultrasound (EUS) evaluation of submucosal masses

F. Gress, C. Schmitt, T. Savides, L. Roubein, N. Nickl, M. Bhutani, B. Hoffman, Douglas Orrick Faigel, W. Wassef, M. Catalane, D. Ciaccia, J. Affronti

Research output: Contribution to journalArticle

Abstract

INTRODUCTION: EUS has been reported to be an accurate means of evaluating and diagnosing submucosal lesions of the GI tract. AIMS OF STUDY: To determine endosonographer interobserver reliability for classifying submucosal masses by EUS. METHODS: Twenty patients with submucosal mass lesions diagnosed at upper endoscopy underwent EUS evaluation. Surgical findings or FNA cytology were available for 16 patients. In 4 patients with obvious cystic/vascular structures (ie, varices) no surgical correlation was necessary. A blinded observer developed a study videotape including critical endoscopic and EUS data for each lesion. The videotape was then distributed to 10 endosonographers with at least one year of experience who then independently reviewed the tape and recorded their diagnosis based upon EUS features. These endosonographers used previously agreed upon standardized EUS diagnostic criteria for each category of lesion (extrinsic compression, leiomyoma, vascular, lipoma, cyst, other submucosal lesion ie; carcinoid) Each endosonographer independently categorized the lesions. A multiple observer kappa (k) statistic for agreement was calculated for each lesion category and an overall k calculated. Kappa accounts for agreement due to chance alone and standardized ranges are available. Log transformation was performed for # cases and kappa and regression analysis used to examine the effect of experience on agreement. RESULTS: We found agreement was excellent for extrinsic compressions, vascular structures and lipomas (k=0.94), good for leiomyoma other submucosal lesions (k=0.67) and fair for cystic lesions. Overall agreement among observers was good (k=0.63) and highly statistically significant (z=21).The lowest kappa scores occurred with the most inexperienced endosonographers and the highest among the most experienced. The association between kappa and experience did not reach statistical significance. CONCLUSION: Interobserver agreement is good for characterizing submucosal masses by EUS. However, it appears to be better for some lesion types than others. Experience may play a role in the successful use of this modality to evaluate submucosal lesions.

Original languageEnglish (US)
JournalGastrointestinal Endoscopy
Volume45
Issue number4
StatePublished - 1997
Externally publishedYes

Fingerprint

Videotape Recording
Lipoma
Blood Vessels
Angiomyoma
Carcinoid Tumor
Varicose Veins
Leiomyoma
Endoscopy
Cell Biology
Gastrointestinal Tract
Cysts
Ultrasonography
Regression Analysis

ASJC Scopus subject areas

  • Gastroenterology

Cite this

Gress, F., Schmitt, C., Savides, T., Roubein, L., Nickl, N., Bhutani, M., ... Affronti, J. (1997). Interobserver agreement among endosonographers for endoscopic ultrasound (EUS) evaluation of submucosal masses. Gastrointestinal Endoscopy, 45(4).

Interobserver agreement among endosonographers for endoscopic ultrasound (EUS) evaluation of submucosal masses. / Gress, F.; Schmitt, C.; Savides, T.; Roubein, L.; Nickl, N.; Bhutani, M.; Hoffman, B.; Faigel, Douglas Orrick; Wassef, W.; Catalane, M.; Ciaccia, D.; Affronti, J.

In: Gastrointestinal Endoscopy, Vol. 45, No. 4, 1997.

Research output: Contribution to journalArticle

Gress, F, Schmitt, C, Savides, T, Roubein, L, Nickl, N, Bhutani, M, Hoffman, B, Faigel, DO, Wassef, W, Catalane, M, Ciaccia, D & Affronti, J 1997, 'Interobserver agreement among endosonographers for endoscopic ultrasound (EUS) evaluation of submucosal masses', Gastrointestinal Endoscopy, vol. 45, no. 4.
Gress, F. ; Schmitt, C. ; Savides, T. ; Roubein, L. ; Nickl, N. ; Bhutani, M. ; Hoffman, B. ; Faigel, Douglas Orrick ; Wassef, W. ; Catalane, M. ; Ciaccia, D. ; Affronti, J. / Interobserver agreement among endosonographers for endoscopic ultrasound (EUS) evaluation of submucosal masses. In: Gastrointestinal Endoscopy. 1997 ; Vol. 45, No. 4.
@article{d2a926d84f474500a4527e65af5e8aa3,
title = "Interobserver agreement among endosonographers for endoscopic ultrasound (EUS) evaluation of submucosal masses",
abstract = "INTRODUCTION: EUS has been reported to be an accurate means of evaluating and diagnosing submucosal lesions of the GI tract. AIMS OF STUDY: To determine endosonographer interobserver reliability for classifying submucosal masses by EUS. METHODS: Twenty patients with submucosal mass lesions diagnosed at upper endoscopy underwent EUS evaluation. Surgical findings or FNA cytology were available for 16 patients. In 4 patients with obvious cystic/vascular structures (ie, varices) no surgical correlation was necessary. A blinded observer developed a study videotape including critical endoscopic and EUS data for each lesion. The videotape was then distributed to 10 endosonographers with at least one year of experience who then independently reviewed the tape and recorded their diagnosis based upon EUS features. These endosonographers used previously agreed upon standardized EUS diagnostic criteria for each category of lesion (extrinsic compression, leiomyoma, vascular, lipoma, cyst, other submucosal lesion ie; carcinoid) Each endosonographer independently categorized the lesions. A multiple observer kappa (k) statistic for agreement was calculated for each lesion category and an overall k calculated. Kappa accounts for agreement due to chance alone and standardized ranges are available. Log transformation was performed for # cases and kappa and regression analysis used to examine the effect of experience on agreement. RESULTS: We found agreement was excellent for extrinsic compressions, vascular structures and lipomas (k=0.94), good for leiomyoma other submucosal lesions (k=0.67) and fair for cystic lesions. Overall agreement among observers was good (k=0.63) and highly statistically significant (z=21).The lowest kappa scores occurred with the most inexperienced endosonographers and the highest among the most experienced. The association between kappa and experience did not reach statistical significance. CONCLUSION: Interobserver agreement is good for characterizing submucosal masses by EUS. However, it appears to be better for some lesion types than others. Experience may play a role in the successful use of this modality to evaluate submucosal lesions.",
author = "F. Gress and C. Schmitt and T. Savides and L. Roubein and N. Nickl and M. Bhutani and B. Hoffman and Faigel, {Douglas Orrick} and W. Wassef and M. Catalane and D. Ciaccia and J. Affronti",
year = "1997",
language = "English (US)",
volume = "45",
journal = "Gastrointestinal Endoscopy",
issn = "0016-5107",
publisher = "Mosby Inc.",
number = "4",

}

TY - JOUR

T1 - Interobserver agreement among endosonographers for endoscopic ultrasound (EUS) evaluation of submucosal masses

AU - Gress, F.

AU - Schmitt, C.

AU - Savides, T.

AU - Roubein, L.

AU - Nickl, N.

AU - Bhutani, M.

AU - Hoffman, B.

AU - Faigel, Douglas Orrick

AU - Wassef, W.

AU - Catalane, M.

AU - Ciaccia, D.

AU - Affronti, J.

PY - 1997

Y1 - 1997

N2 - INTRODUCTION: EUS has been reported to be an accurate means of evaluating and diagnosing submucosal lesions of the GI tract. AIMS OF STUDY: To determine endosonographer interobserver reliability for classifying submucosal masses by EUS. METHODS: Twenty patients with submucosal mass lesions diagnosed at upper endoscopy underwent EUS evaluation. Surgical findings or FNA cytology were available for 16 patients. In 4 patients with obvious cystic/vascular structures (ie, varices) no surgical correlation was necessary. A blinded observer developed a study videotape including critical endoscopic and EUS data for each lesion. The videotape was then distributed to 10 endosonographers with at least one year of experience who then independently reviewed the tape and recorded their diagnosis based upon EUS features. These endosonographers used previously agreed upon standardized EUS diagnostic criteria for each category of lesion (extrinsic compression, leiomyoma, vascular, lipoma, cyst, other submucosal lesion ie; carcinoid) Each endosonographer independently categorized the lesions. A multiple observer kappa (k) statistic for agreement was calculated for each lesion category and an overall k calculated. Kappa accounts for agreement due to chance alone and standardized ranges are available. Log transformation was performed for # cases and kappa and regression analysis used to examine the effect of experience on agreement. RESULTS: We found agreement was excellent for extrinsic compressions, vascular structures and lipomas (k=0.94), good for leiomyoma other submucosal lesions (k=0.67) and fair for cystic lesions. Overall agreement among observers was good (k=0.63) and highly statistically significant (z=21).The lowest kappa scores occurred with the most inexperienced endosonographers and the highest among the most experienced. The association between kappa and experience did not reach statistical significance. CONCLUSION: Interobserver agreement is good for characterizing submucosal masses by EUS. However, it appears to be better for some lesion types than others. Experience may play a role in the successful use of this modality to evaluate submucosal lesions.

AB - INTRODUCTION: EUS has been reported to be an accurate means of evaluating and diagnosing submucosal lesions of the GI tract. AIMS OF STUDY: To determine endosonographer interobserver reliability for classifying submucosal masses by EUS. METHODS: Twenty patients with submucosal mass lesions diagnosed at upper endoscopy underwent EUS evaluation. Surgical findings or FNA cytology were available for 16 patients. In 4 patients with obvious cystic/vascular structures (ie, varices) no surgical correlation was necessary. A blinded observer developed a study videotape including critical endoscopic and EUS data for each lesion. The videotape was then distributed to 10 endosonographers with at least one year of experience who then independently reviewed the tape and recorded their diagnosis based upon EUS features. These endosonographers used previously agreed upon standardized EUS diagnostic criteria for each category of lesion (extrinsic compression, leiomyoma, vascular, lipoma, cyst, other submucosal lesion ie; carcinoid) Each endosonographer independently categorized the lesions. A multiple observer kappa (k) statistic for agreement was calculated for each lesion category and an overall k calculated. Kappa accounts for agreement due to chance alone and standardized ranges are available. Log transformation was performed for # cases and kappa and regression analysis used to examine the effect of experience on agreement. RESULTS: We found agreement was excellent for extrinsic compressions, vascular structures and lipomas (k=0.94), good for leiomyoma other submucosal lesions (k=0.67) and fair for cystic lesions. Overall agreement among observers was good (k=0.63) and highly statistically significant (z=21).The lowest kappa scores occurred with the most inexperienced endosonographers and the highest among the most experienced. The association between kappa and experience did not reach statistical significance. CONCLUSION: Interobserver agreement is good for characterizing submucosal masses by EUS. However, it appears to be better for some lesion types than others. Experience may play a role in the successful use of this modality to evaluate submucosal lesions.

UR - http://www.scopus.com/inward/record.url?scp=33748958524&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33748958524&partnerID=8YFLogxK

M3 - Article

AN - SCOPUS:33748958524

VL - 45

JO - Gastrointestinal Endoscopy

JF - Gastrointestinal Endoscopy

SN - 0016-5107

IS - 4

ER -