Interobserver variability of sonographic features used in the American college of radiology thyroid imaging reporting and data system

Jenny K. Hoang, William D. Middleton, Alfredo E. Farjat, Sharlene A. Teefey, Nicole Abinanti, Fernando J. Boschini, Abraham J. Bronner, Nirvikar Dahiya, Barbara S. Hertzberg, Justin R. Newman, Daniel Scanga, Robert C. Vogler, Franklin N. Tessler

Research output: Contribution to journalArticle

12 Citations (Scopus)

Abstract

OBJECTIVE. The purpose of this study was to assess interobserver variability in assigning features in the American College of Radiology Thyroid Imaging Reporting and Data System (ACR TI-RADS) lexicon and in making recommendations for thyroid nodule biopsy. MATERIALS AND METHODS. The study cohort comprised 100 nodules in 92 patients who underwent fine-needle aspiration with definitive cytologic results (Bethesda category II or VI) or diagnostic lobectomy between April 2009 and May 2010. Eight board-certified radiologists evaluated the nodules according to the five feature categories that constitute ACR TI-RADS and gave a biopsy recommendation based on their own practice. Variability in feature assignment and biopsy recommendation was assessed with the Fleiss kappa statistic. RESULTS. Agreement in interpretation was fair to moderate for all features except shape (κ = 0.61) and macrocalcifications (κ = 0.73), which had substantial agreement. The features with the poorest agreement were margin and other types of echogenic foci, which had kappa values ranging from 0.25 to 0.39, indicating fair agreement. Interobserver agreement regarding biopsy recommendation was fair (κ = 0.22) based on radiologists’ current practice. Applying ACR TI-RADS resulted in moderate agreement (κ = 0.51). CONCLUSION. Variability in interpreting thyroid nodule sonographic features was highest for margin and all types of echogenic foci, except for macrocalcifications. Because radiologists’ interpretations of these features change the level of suspicion of thyroid malignancy, the results of this study suggest a need for further education. Despite the variability in assigning features, adoption of ACR TI-RADS improves agreement for recommending biopsy.

Original languageEnglish (US)
Pages (from-to)162-167
Number of pages6
JournalAmerican Journal of Roentgenology
Volume211
Issue number1
DOIs
StatePublished - Jul 1 2018

Fingerprint

Observer Variation
Radiology
Information Systems
Thyroid Gland
Biopsy
Thyroid Nodule
Fine Needle Biopsy
Cohort Studies
Education
Radiologists
Neoplasms

Keywords

  • Interobserver agreement
  • Interobserver variability
  • Thyroid malignancy
  • Thyroid ultrasound
  • TI-RADS

ASJC Scopus subject areas

  • Radiology Nuclear Medicine and imaging

Cite this

Hoang, J. K., Middleton, W. D., Farjat, A. E., Teefey, S. A., Abinanti, N., Boschini, F. J., ... Tessler, F. N. (2018). Interobserver variability of sonographic features used in the American college of radiology thyroid imaging reporting and data system. American Journal of Roentgenology, 211(1), 162-167. https://doi.org/10.2214/AJR.17.19192

Interobserver variability of sonographic features used in the American college of radiology thyroid imaging reporting and data system. / Hoang, Jenny K.; Middleton, William D.; Farjat, Alfredo E.; Teefey, Sharlene A.; Abinanti, Nicole; Boschini, Fernando J.; Bronner, Abraham J.; Dahiya, Nirvikar; Hertzberg, Barbara S.; Newman, Justin R.; Scanga, Daniel; Vogler, Robert C.; Tessler, Franklin N.

In: American Journal of Roentgenology, Vol. 211, No. 1, 01.07.2018, p. 162-167.

Research output: Contribution to journalArticle

Hoang, JK, Middleton, WD, Farjat, AE, Teefey, SA, Abinanti, N, Boschini, FJ, Bronner, AJ, Dahiya, N, Hertzberg, BS, Newman, JR, Scanga, D, Vogler, RC & Tessler, FN 2018, 'Interobserver variability of sonographic features used in the American college of radiology thyroid imaging reporting and data system', American Journal of Roentgenology, vol. 211, no. 1, pp. 162-167. https://doi.org/10.2214/AJR.17.19192
Hoang, Jenny K. ; Middleton, William D. ; Farjat, Alfredo E. ; Teefey, Sharlene A. ; Abinanti, Nicole ; Boschini, Fernando J. ; Bronner, Abraham J. ; Dahiya, Nirvikar ; Hertzberg, Barbara S. ; Newman, Justin R. ; Scanga, Daniel ; Vogler, Robert C. ; Tessler, Franklin N. / Interobserver variability of sonographic features used in the American college of radiology thyroid imaging reporting and data system. In: American Journal of Roentgenology. 2018 ; Vol. 211, No. 1. pp. 162-167.
@article{e0cba56ff3244344b6e83879c2c4b79d,
title = "Interobserver variability of sonographic features used in the American college of radiology thyroid imaging reporting and data system",
abstract = "OBJECTIVE. The purpose of this study was to assess interobserver variability in assigning features in the American College of Radiology Thyroid Imaging Reporting and Data System (ACR TI-RADS) lexicon and in making recommendations for thyroid nodule biopsy. MATERIALS AND METHODS. The study cohort comprised 100 nodules in 92 patients who underwent fine-needle aspiration with definitive cytologic results (Bethesda category II or VI) or diagnostic lobectomy between April 2009 and May 2010. Eight board-certified radiologists evaluated the nodules according to the five feature categories that constitute ACR TI-RADS and gave a biopsy recommendation based on their own practice. Variability in feature assignment and biopsy recommendation was assessed with the Fleiss kappa statistic. RESULTS. Agreement in interpretation was fair to moderate for all features except shape (κ = 0.61) and macrocalcifications (κ = 0.73), which had substantial agreement. The features with the poorest agreement were margin and other types of echogenic foci, which had kappa values ranging from 0.25 to 0.39, indicating fair agreement. Interobserver agreement regarding biopsy recommendation was fair (κ = 0.22) based on radiologists’ current practice. Applying ACR TI-RADS resulted in moderate agreement (κ = 0.51). CONCLUSION. Variability in interpreting thyroid nodule sonographic features was highest for margin and all types of echogenic foci, except for macrocalcifications. Because radiologists’ interpretations of these features change the level of suspicion of thyroid malignancy, the results of this study suggest a need for further education. Despite the variability in assigning features, adoption of ACR TI-RADS improves agreement for recommending biopsy.",
keywords = "Interobserver agreement, Interobserver variability, Thyroid malignancy, Thyroid ultrasound, TI-RADS",
author = "Hoang, {Jenny K.} and Middleton, {William D.} and Farjat, {Alfredo E.} and Teefey, {Sharlene A.} and Nicole Abinanti and Boschini, {Fernando J.} and Bronner, {Abraham J.} and Nirvikar Dahiya and Hertzberg, {Barbara S.} and Newman, {Justin R.} and Daniel Scanga and Vogler, {Robert C.} and Tessler, {Franklin N.}",
year = "2018",
month = "7",
day = "1",
doi = "10.2214/AJR.17.19192",
language = "English (US)",
volume = "211",
pages = "162--167",
journal = "American Journal of Roentgenology",
issn = "0361-803X",
publisher = "American Roentgen Ray Society",
number = "1",

}

TY - JOUR

T1 - Interobserver variability of sonographic features used in the American college of radiology thyroid imaging reporting and data system

AU - Hoang, Jenny K.

AU - Middleton, William D.

AU - Farjat, Alfredo E.

AU - Teefey, Sharlene A.

AU - Abinanti, Nicole

AU - Boschini, Fernando J.

AU - Bronner, Abraham J.

AU - Dahiya, Nirvikar

AU - Hertzberg, Barbara S.

AU - Newman, Justin R.

AU - Scanga, Daniel

AU - Vogler, Robert C.

AU - Tessler, Franklin N.

PY - 2018/7/1

Y1 - 2018/7/1

N2 - OBJECTIVE. The purpose of this study was to assess interobserver variability in assigning features in the American College of Radiology Thyroid Imaging Reporting and Data System (ACR TI-RADS) lexicon and in making recommendations for thyroid nodule biopsy. MATERIALS AND METHODS. The study cohort comprised 100 nodules in 92 patients who underwent fine-needle aspiration with definitive cytologic results (Bethesda category II or VI) or diagnostic lobectomy between April 2009 and May 2010. Eight board-certified radiologists evaluated the nodules according to the five feature categories that constitute ACR TI-RADS and gave a biopsy recommendation based on their own practice. Variability in feature assignment and biopsy recommendation was assessed with the Fleiss kappa statistic. RESULTS. Agreement in interpretation was fair to moderate for all features except shape (κ = 0.61) and macrocalcifications (κ = 0.73), which had substantial agreement. The features with the poorest agreement were margin and other types of echogenic foci, which had kappa values ranging from 0.25 to 0.39, indicating fair agreement. Interobserver agreement regarding biopsy recommendation was fair (κ = 0.22) based on radiologists’ current practice. Applying ACR TI-RADS resulted in moderate agreement (κ = 0.51). CONCLUSION. Variability in interpreting thyroid nodule sonographic features was highest for margin and all types of echogenic foci, except for macrocalcifications. Because radiologists’ interpretations of these features change the level of suspicion of thyroid malignancy, the results of this study suggest a need for further education. Despite the variability in assigning features, adoption of ACR TI-RADS improves agreement for recommending biopsy.

AB - OBJECTIVE. The purpose of this study was to assess interobserver variability in assigning features in the American College of Radiology Thyroid Imaging Reporting and Data System (ACR TI-RADS) lexicon and in making recommendations for thyroid nodule biopsy. MATERIALS AND METHODS. The study cohort comprised 100 nodules in 92 patients who underwent fine-needle aspiration with definitive cytologic results (Bethesda category II or VI) or diagnostic lobectomy between April 2009 and May 2010. Eight board-certified radiologists evaluated the nodules according to the five feature categories that constitute ACR TI-RADS and gave a biopsy recommendation based on their own practice. Variability in feature assignment and biopsy recommendation was assessed with the Fleiss kappa statistic. RESULTS. Agreement in interpretation was fair to moderate for all features except shape (κ = 0.61) and macrocalcifications (κ = 0.73), which had substantial agreement. The features with the poorest agreement were margin and other types of echogenic foci, which had kappa values ranging from 0.25 to 0.39, indicating fair agreement. Interobserver agreement regarding biopsy recommendation was fair (κ = 0.22) based on radiologists’ current practice. Applying ACR TI-RADS resulted in moderate agreement (κ = 0.51). CONCLUSION. Variability in interpreting thyroid nodule sonographic features was highest for margin and all types of echogenic foci, except for macrocalcifications. Because radiologists’ interpretations of these features change the level of suspicion of thyroid malignancy, the results of this study suggest a need for further education. Despite the variability in assigning features, adoption of ACR TI-RADS improves agreement for recommending biopsy.

KW - Interobserver agreement

KW - Interobserver variability

KW - Thyroid malignancy

KW - Thyroid ultrasound

KW - TI-RADS

UR - http://www.scopus.com/inward/record.url?scp=85049153732&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85049153732&partnerID=8YFLogxK

U2 - 10.2214/AJR.17.19192

DO - 10.2214/AJR.17.19192

M3 - Article

VL - 211

SP - 162

EP - 167

JO - American Journal of Roentgenology

JF - American Journal of Roentgenology

SN - 0361-803X

IS - 1

ER -