Correlation between a deep-learning-based model observer and human observer for a realistic lung nodule localization task in chest CT

Hao Gong, Andrew Walther, Qiyuan Hu, Chi Wan Koo, Edwin A. Takahashi, David L. Levin, Tucker Johnson, Megan J. Hora, Shuai Leng, J. G. Fletcher, Cynthia H McCollough, Lifeng Yu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Mathematical model observers (MOs) have become popular in task-based CT image quality assessment, since, once proven to be correlated with human observers (HOs), these MOs can be used to estimate HO performance. However, typical MO studies are limited to phantom data which only involve uniform background. In practice, anatomical background variability and tissue non-uniformity affect HO lesion detection performance. Recently, we have proposed a deep-learning-based MO (DL-MO). In this study, we aim to investigate the correlation between this DL-MO and HOs for a lung-nodule localization task in chest CT. Using a patient database that contains 50 lung cancer screening CT patient cases, 12 different experimental conditions were generated, including 4 radiation dose levels, 3 nodule sizes, 2 nodule types and 3 reconstruction types. These conditions were created by using a validated noise and lesion insertion tool. Four subspecialized radiologists performed the HO study for all 12 conditions individually in a randomized fashion. The DL-MO was trained and tested for the same dataset. The performance of DL-MO and HO was compared across all the experimental conditions. DL-MO performance was strongly correlated with HO performance (Pearson's correlation coefficient: 0.988 with a 95% confidence interval of [0.894, 0.999]). These results demonstrate the potential to use the proposed DL-MO to predict HO performance for the task of lung nodule localization in chest CT.

Original languageEnglish (US)
Title of host publicationMedical Imaging 2019
Subtitle of host publicationImage Perception, Observer Performance, and Technology Assessment
EditorsRobert M. Nishikawa, Frank W. Samuelson
PublisherSPIE
ISBN (Electronic)9781510625518
DOIs
StatePublished - Jan 1 2019
EventMedical Imaging 2019: Image Perception, Observer Performance, and Technology Assessment - San Diego, United States
Duration: Feb 20 2019Feb 21 2019

Publication series

NameProgress in Biomedical Optics and Imaging - Proceedings of SPIE
Volume10952
ISSN (Print)1605-7422

Conference

ConferenceMedical Imaging 2019: Image Perception, Observer Performance, and Technology Assessment
CountryUnited States
CitySan Diego
Period2/20/192/21/19

Fingerprint

chest
nodules
lungs
learning
Thorax
Learning
Lung
lesions
correlation coefficients
nonuniformity
Image quality
Dosimetry
Deep learning
insertion
confidence
mathematical models
Screening
Task Performance and Analysis
screening
cancer

Keywords

  • Deep learning
  • Lung nodule detection
  • Model observer
  • Partial least square regression
  • Taskbased image quality assessment
  • X-ray CT

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Atomic and Molecular Physics, and Optics
  • Biomaterials
  • Radiology Nuclear Medicine and imaging

Cite this

Gong, H., Walther, A., Hu, Q., Koo, C. W., Takahashi, E. A., Levin, D. L., ... Yu, L. (2019). Correlation between a deep-learning-based model observer and human observer for a realistic lung nodule localization task in chest CT. In R. M. Nishikawa, & F. W. Samuelson (Eds.), Medical Imaging 2019: Image Perception, Observer Performance, and Technology Assessment [109520K] (Progress in Biomedical Optics and Imaging - Proceedings of SPIE; Vol. 10952). SPIE. https://doi.org/10.1117/12.2513451

Correlation between a deep-learning-based model observer and human observer for a realistic lung nodule localization task in chest CT. / Gong, Hao; Walther, Andrew; Hu, Qiyuan; Koo, Chi Wan; Takahashi, Edwin A.; Levin, David L.; Johnson, Tucker; Hora, Megan J.; Leng, Shuai; Fletcher, J. G.; McCollough, Cynthia H; Yu, Lifeng.

Medical Imaging 2019: Image Perception, Observer Performance, and Technology Assessment. ed. / Robert M. Nishikawa; Frank W. Samuelson. SPIE, 2019. 109520K (Progress in Biomedical Optics and Imaging - Proceedings of SPIE; Vol. 10952).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Gong, H, Walther, A, Hu, Q, Koo, CW, Takahashi, EA, Levin, DL, Johnson, T, Hora, MJ, Leng, S, Fletcher, JG, McCollough, CH & Yu, L 2019, Correlation between a deep-learning-based model observer and human observer for a realistic lung nodule localization task in chest CT. in RM Nishikawa & FW Samuelson (eds), Medical Imaging 2019: Image Perception, Observer Performance, and Technology Assessment., 109520K, Progress in Biomedical Optics and Imaging - Proceedings of SPIE, vol. 10952, SPIE, Medical Imaging 2019: Image Perception, Observer Performance, and Technology Assessment, San Diego, United States, 2/20/19. https://doi.org/10.1117/12.2513451
Gong H, Walther A, Hu Q, Koo CW, Takahashi EA, Levin DL et al. Correlation between a deep-learning-based model observer and human observer for a realistic lung nodule localization task in chest CT. In Nishikawa RM, Samuelson FW, editors, Medical Imaging 2019: Image Perception, Observer Performance, and Technology Assessment. SPIE. 2019. 109520K. (Progress in Biomedical Optics and Imaging - Proceedings of SPIE). https://doi.org/10.1117/12.2513451
Gong, Hao ; Walther, Andrew ; Hu, Qiyuan ; Koo, Chi Wan ; Takahashi, Edwin A. ; Levin, David L. ; Johnson, Tucker ; Hora, Megan J. ; Leng, Shuai ; Fletcher, J. G. ; McCollough, Cynthia H ; Yu, Lifeng. / Correlation between a deep-learning-based model observer and human observer for a realistic lung nodule localization task in chest CT. Medical Imaging 2019: Image Perception, Observer Performance, and Technology Assessment. editor / Robert M. Nishikawa ; Frank W. Samuelson. SPIE, 2019. (Progress in Biomedical Optics and Imaging - Proceedings of SPIE).
@inproceedings{ce87c2acd6824482af5283707bdb3f13,
title = "Correlation between a deep-learning-based model observer and human observer for a realistic lung nodule localization task in chest CT",
abstract = "Mathematical model observers (MOs) have become popular in task-based CT image quality assessment, since, once proven to be correlated with human observers (HOs), these MOs can be used to estimate HO performance. However, typical MO studies are limited to phantom data which only involve uniform background. In practice, anatomical background variability and tissue non-uniformity affect HO lesion detection performance. Recently, we have proposed a deep-learning-based MO (DL-MO). In this study, we aim to investigate the correlation between this DL-MO and HOs for a lung-nodule localization task in chest CT. Using a patient database that contains 50 lung cancer screening CT patient cases, 12 different experimental conditions were generated, including 4 radiation dose levels, 3 nodule sizes, 2 nodule types and 3 reconstruction types. These conditions were created by using a validated noise and lesion insertion tool. Four subspecialized radiologists performed the HO study for all 12 conditions individually in a randomized fashion. The DL-MO was trained and tested for the same dataset. The performance of DL-MO and HO was compared across all the experimental conditions. DL-MO performance was strongly correlated with HO performance (Pearson's correlation coefficient: 0.988 with a 95{\%} confidence interval of [0.894, 0.999]). These results demonstrate the potential to use the proposed DL-MO to predict HO performance for the task of lung nodule localization in chest CT.",
keywords = "Deep learning, Lung nodule detection, Model observer, Partial least square regression, Taskbased image quality assessment, X-ray CT",
author = "Hao Gong and Andrew Walther and Qiyuan Hu and Koo, {Chi Wan} and Takahashi, {Edwin A.} and Levin, {David L.} and Tucker Johnson and Hora, {Megan J.} and Shuai Leng and Fletcher, {J. G.} and McCollough, {Cynthia H} and Lifeng Yu",
year = "2019",
month = "1",
day = "1",
doi = "10.1117/12.2513451",
language = "English (US)",
series = "Progress in Biomedical Optics and Imaging - Proceedings of SPIE",
publisher = "SPIE",
editor = "Nishikawa, {Robert M.} and Samuelson, {Frank W.}",
booktitle = "Medical Imaging 2019",

}

TY - GEN

T1 - Correlation between a deep-learning-based model observer and human observer for a realistic lung nodule localization task in chest CT

AU - Gong, Hao

AU - Walther, Andrew

AU - Hu, Qiyuan

AU - Koo, Chi Wan

AU - Takahashi, Edwin A.

AU - Levin, David L.

AU - Johnson, Tucker

AU - Hora, Megan J.

AU - Leng, Shuai

AU - Fletcher, J. G.

AU - McCollough, Cynthia H

AU - Yu, Lifeng

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Mathematical model observers (MOs) have become popular in task-based CT image quality assessment, since, once proven to be correlated with human observers (HOs), these MOs can be used to estimate HO performance. However, typical MO studies are limited to phantom data which only involve uniform background. In practice, anatomical background variability and tissue non-uniformity affect HO lesion detection performance. Recently, we have proposed a deep-learning-based MO (DL-MO). In this study, we aim to investigate the correlation between this DL-MO and HOs for a lung-nodule localization task in chest CT. Using a patient database that contains 50 lung cancer screening CT patient cases, 12 different experimental conditions were generated, including 4 radiation dose levels, 3 nodule sizes, 2 nodule types and 3 reconstruction types. These conditions were created by using a validated noise and lesion insertion tool. Four subspecialized radiologists performed the HO study for all 12 conditions individually in a randomized fashion. The DL-MO was trained and tested for the same dataset. The performance of DL-MO and HO was compared across all the experimental conditions. DL-MO performance was strongly correlated with HO performance (Pearson's correlation coefficient: 0.988 with a 95% confidence interval of [0.894, 0.999]). These results demonstrate the potential to use the proposed DL-MO to predict HO performance for the task of lung nodule localization in chest CT.

AB - Mathematical model observers (MOs) have become popular in task-based CT image quality assessment, since, once proven to be correlated with human observers (HOs), these MOs can be used to estimate HO performance. However, typical MO studies are limited to phantom data which only involve uniform background. In practice, anatomical background variability and tissue non-uniformity affect HO lesion detection performance. Recently, we have proposed a deep-learning-based MO (DL-MO). In this study, we aim to investigate the correlation between this DL-MO and HOs for a lung-nodule localization task in chest CT. Using a patient database that contains 50 lung cancer screening CT patient cases, 12 different experimental conditions were generated, including 4 radiation dose levels, 3 nodule sizes, 2 nodule types and 3 reconstruction types. These conditions were created by using a validated noise and lesion insertion tool. Four subspecialized radiologists performed the HO study for all 12 conditions individually in a randomized fashion. The DL-MO was trained and tested for the same dataset. The performance of DL-MO and HO was compared across all the experimental conditions. DL-MO performance was strongly correlated with HO performance (Pearson's correlation coefficient: 0.988 with a 95% confidence interval of [0.894, 0.999]). These results demonstrate the potential to use the proposed DL-MO to predict HO performance for the task of lung nodule localization in chest CT.

KW - Deep learning

KW - Lung nodule detection

KW - Model observer

KW - Partial least square regression

KW - Taskbased image quality assessment

KW - X-ray CT

UR - http://www.scopus.com/inward/record.url?scp=85068693821&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85068693821&partnerID=8YFLogxK

U2 - 10.1117/12.2513451

DO - 10.1117/12.2513451

M3 - Conference contribution

AN - SCOPUS:85068693821

T3 - Progress in Biomedical Optics and Imaging - Proceedings of SPIE

BT - Medical Imaging 2019

A2 - Nishikawa, Robert M.

A2 - Samuelson, Frank W.

PB - SPIE

ER -