Assessing the inter-observer variability of Computer-Aided Nodule Assessment and Risk Yield (CANARY) to characterize lung adenocarcinomas

Erica C. Nakajima, Michael P. Frankland, Tucker Johnson, Sanja L. Antic, Heidi Chen, Sheau Chiann Chen, Ronald A. Karwoski, Ronald Walker, Bennett A. Landman, Ryan D. Clay, Brian Jack Bartholmai, Srinivasan Rajagopalan, Tobias D Peikert, Pierre P. Massion, Fabien Maldonado

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

Lung adenocarcinoma (ADC), the most common lung cancer type, is recognized increasingly as a disease spectrum. To guide individualized patient care, a non-invasive means of distinguishing indolent from aggressive ADC subtypes is needed urgently. Computer-Aided Nodule Assessment and Risk Yield (CANARY) is a novel computed tomography (CT) tool that characterizes early ADCs by detecting nine distinct CT voxel classes, representing a spectrum of lepidic to invasive growth, within an ADC. CANARY characterization has been shown to correlate with ADC histology and patient outcomes. This study evaluated the inter-observer variability of CANARY analysis. Three novice observers segmented and analyzed independently 95 biopsy-confirmed lung ADCs from Vanderbilt University Medical Center/ Nashville Veterans Administration Tennessee Valley Healthcare system (VUMC/TVHS) and the Mayo Clinic (Mayo). Inter-observer variability was measured using intra-class correlation coefficient (ICC). The average ICC for all CANARY classes was 0.828 (95% CI 0.76, 0.895) for the VUMC/TVHS cohort, and 0.852 (95% CI 0.804, 0.901) for the Mayo cohort. The most invasive voxel classes had the highest ICC values. To determine whether nodule size influenced inter-observer variability, an additional cohort of 49 sub-centimeter nodules from Mayo were also segmented by three observers, with similar ICC results. Our study demonstrates that CANARY ADC classification between novice CANARY users has an acceptably low degree of variability, and supports the further development of CANARY for clinical application.

Original languageEnglish (US)
Article numbere0198118
JournalPLoS One
Volume13
Issue number6
DOIs
StatePublished - Jun 1 2018

Fingerprint

Observer Variation
adenocarcinoma
lungs
Adenocarcinoma
veterans
United States Department of Veterans Affairs
computed tomography
health services
Tomography
valleys
Delivery of Health Care
Histology
patient care
Biopsy
lung neoplasms
Adenocarcinoma of lung
histology
biopsy
Lung Neoplasms
Patient Care

ASJC Scopus subject areas

  • Biochemistry, Genetics and Molecular Biology(all)
  • Agricultural and Biological Sciences(all)

Cite this

Assessing the inter-observer variability of Computer-Aided Nodule Assessment and Risk Yield (CANARY) to characterize lung adenocarcinomas. / Nakajima, Erica C.; Frankland, Michael P.; Johnson, Tucker; Antic, Sanja L.; Chen, Heidi; Chen, Sheau Chiann; Karwoski, Ronald A.; Walker, Ronald; Landman, Bennett A.; Clay, Ryan D.; Bartholmai, Brian Jack; Rajagopalan, Srinivasan; Peikert, Tobias D; Massion, Pierre P.; Maldonado, Fabien.

In: PLoS One, Vol. 13, No. 6, e0198118, 01.06.2018.

Research output: Contribution to journalArticle

Nakajima, EC, Frankland, MP, Johnson, T, Antic, SL, Chen, H, Chen, SC, Karwoski, RA, Walker, R, Landman, BA, Clay, RD, Bartholmai, BJ, Rajagopalan, S, Peikert, TD, Massion, PP & Maldonado, F 2018, 'Assessing the inter-observer variability of Computer-Aided Nodule Assessment and Risk Yield (CANARY) to characterize lung adenocarcinomas', PLoS One, vol. 13, no. 6, e0198118. https://doi.org/10.1371/journal.pone.0198118
Nakajima, Erica C. ; Frankland, Michael P. ; Johnson, Tucker ; Antic, Sanja L. ; Chen, Heidi ; Chen, Sheau Chiann ; Karwoski, Ronald A. ; Walker, Ronald ; Landman, Bennett A. ; Clay, Ryan D. ; Bartholmai, Brian Jack ; Rajagopalan, Srinivasan ; Peikert, Tobias D ; Massion, Pierre P. ; Maldonado, Fabien. / Assessing the inter-observer variability of Computer-Aided Nodule Assessment and Risk Yield (CANARY) to characterize lung adenocarcinomas. In: PLoS One. 2018 ; Vol. 13, No. 6.
@article{fe0f5489084f43dc8eab92ef74fbf07f,
title = "Assessing the inter-observer variability of Computer-Aided Nodule Assessment and Risk Yield (CANARY) to characterize lung adenocarcinomas",
abstract = "Lung adenocarcinoma (ADC), the most common lung cancer type, is recognized increasingly as a disease spectrum. To guide individualized patient care, a non-invasive means of distinguishing indolent from aggressive ADC subtypes is needed urgently. Computer-Aided Nodule Assessment and Risk Yield (CANARY) is a novel computed tomography (CT) tool that characterizes early ADCs by detecting nine distinct CT voxel classes, representing a spectrum of lepidic to invasive growth, within an ADC. CANARY characterization has been shown to correlate with ADC histology and patient outcomes. This study evaluated the inter-observer variability of CANARY analysis. Three novice observers segmented and analyzed independently 95 biopsy-confirmed lung ADCs from Vanderbilt University Medical Center/ Nashville Veterans Administration Tennessee Valley Healthcare system (VUMC/TVHS) and the Mayo Clinic (Mayo). Inter-observer variability was measured using intra-class correlation coefficient (ICC). The average ICC for all CANARY classes was 0.828 (95{\%} CI 0.76, 0.895) for the VUMC/TVHS cohort, and 0.852 (95{\%} CI 0.804, 0.901) for the Mayo cohort. The most invasive voxel classes had the highest ICC values. To determine whether nodule size influenced inter-observer variability, an additional cohort of 49 sub-centimeter nodules from Mayo were also segmented by three observers, with similar ICC results. Our study demonstrates that CANARY ADC classification between novice CANARY users has an acceptably low degree of variability, and supports the further development of CANARY for clinical application.",
author = "Nakajima, {Erica C.} and Frankland, {Michael P.} and Tucker Johnson and Antic, {Sanja L.} and Heidi Chen and Chen, {Sheau Chiann} and Karwoski, {Ronald A.} and Ronald Walker and Landman, {Bennett A.} and Clay, {Ryan D.} and Bartholmai, {Brian Jack} and Srinivasan Rajagopalan and Peikert, {Tobias D} and Massion, {Pierre P.} and Fabien Maldonado",
year = "2018",
month = "6",
day = "1",
doi = "10.1371/journal.pone.0198118",
language = "English (US)",
volume = "13",
journal = "PLoS One",
issn = "1932-6203",
publisher = "Public Library of Science",
number = "6",

}

TY - JOUR

T1 - Assessing the inter-observer variability of Computer-Aided Nodule Assessment and Risk Yield (CANARY) to characterize lung adenocarcinomas

AU - Nakajima, Erica C.

AU - Frankland, Michael P.

AU - Johnson, Tucker

AU - Antic, Sanja L.

AU - Chen, Heidi

AU - Chen, Sheau Chiann

AU - Karwoski, Ronald A.

AU - Walker, Ronald

AU - Landman, Bennett A.

AU - Clay, Ryan D.

AU - Bartholmai, Brian Jack

AU - Rajagopalan, Srinivasan

AU - Peikert, Tobias D

AU - Massion, Pierre P.

AU - Maldonado, Fabien

PY - 2018/6/1

Y1 - 2018/6/1

N2 - Lung adenocarcinoma (ADC), the most common lung cancer type, is recognized increasingly as a disease spectrum. To guide individualized patient care, a non-invasive means of distinguishing indolent from aggressive ADC subtypes is needed urgently. Computer-Aided Nodule Assessment and Risk Yield (CANARY) is a novel computed tomography (CT) tool that characterizes early ADCs by detecting nine distinct CT voxel classes, representing a spectrum of lepidic to invasive growth, within an ADC. CANARY characterization has been shown to correlate with ADC histology and patient outcomes. This study evaluated the inter-observer variability of CANARY analysis. Three novice observers segmented and analyzed independently 95 biopsy-confirmed lung ADCs from Vanderbilt University Medical Center/ Nashville Veterans Administration Tennessee Valley Healthcare system (VUMC/TVHS) and the Mayo Clinic (Mayo). Inter-observer variability was measured using intra-class correlation coefficient (ICC). The average ICC for all CANARY classes was 0.828 (95% CI 0.76, 0.895) for the VUMC/TVHS cohort, and 0.852 (95% CI 0.804, 0.901) for the Mayo cohort. The most invasive voxel classes had the highest ICC values. To determine whether nodule size influenced inter-observer variability, an additional cohort of 49 sub-centimeter nodules from Mayo were also segmented by three observers, with similar ICC results. Our study demonstrates that CANARY ADC classification between novice CANARY users has an acceptably low degree of variability, and supports the further development of CANARY for clinical application.

AB - Lung adenocarcinoma (ADC), the most common lung cancer type, is recognized increasingly as a disease spectrum. To guide individualized patient care, a non-invasive means of distinguishing indolent from aggressive ADC subtypes is needed urgently. Computer-Aided Nodule Assessment and Risk Yield (CANARY) is a novel computed tomography (CT) tool that characterizes early ADCs by detecting nine distinct CT voxel classes, representing a spectrum of lepidic to invasive growth, within an ADC. CANARY characterization has been shown to correlate with ADC histology and patient outcomes. This study evaluated the inter-observer variability of CANARY analysis. Three novice observers segmented and analyzed independently 95 biopsy-confirmed lung ADCs from Vanderbilt University Medical Center/ Nashville Veterans Administration Tennessee Valley Healthcare system (VUMC/TVHS) and the Mayo Clinic (Mayo). Inter-observer variability was measured using intra-class correlation coefficient (ICC). The average ICC for all CANARY classes was 0.828 (95% CI 0.76, 0.895) for the VUMC/TVHS cohort, and 0.852 (95% CI 0.804, 0.901) for the Mayo cohort. The most invasive voxel classes had the highest ICC values. To determine whether nodule size influenced inter-observer variability, an additional cohort of 49 sub-centimeter nodules from Mayo were also segmented by three observers, with similar ICC results. Our study demonstrates that CANARY ADC classification between novice CANARY users has an acceptably low degree of variability, and supports the further development of CANARY for clinical application.

UR - http://www.scopus.com/inward/record.url?scp=85048058104&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85048058104&partnerID=8YFLogxK

U2 - 10.1371/journal.pone.0198118

DO - 10.1371/journal.pone.0198118

M3 - Article

C2 - 29856852

AN - SCOPUS:85048058104

VL - 13

JO - PLoS One

JF - PLoS One

SN - 1932-6203

IS - 6

M1 - e0198118

ER -