Medical image synthesis for data augmentation and anonymization using generative adversarial networks

Hoo Chang Shin, Neil A. Tenenholtz, Jameson K. Rogers, Christopher Schwarz, Matthew L. Senjem, Jeffrey L. Gunter, Katherine P. Andriole, Mark Michalski

Research output: Chapter in Book/Report/Conference proceedingConference contribution

22 Citations (Scopus)

Abstract

Data diversity is critical to success when training deep learning models. Medical imaging data sets are often imbalanced as pathologic findings are generally rare, which introduces significant challenges when training deep learning models. In this work, we propose a method to generate synthetic abnormal MRI images with brain tumors by training a generative adversarial network using two publicly available data sets of brain MRI. We demonstrate two unique benefits that the synthetic images provide. First, we illustrate improved performance on tumor segmentation by leveraging the synthetic images as a form of data augmentation. Second, we demonstrate the value of generative models as an anonymization tool, achieving comparable tumor segmentation results when trained on the synthetic data versus when trained on real subject data. Together, these results offer a potential solution to two of the largest challenges facing machine learning in medical imaging, namely the small incidence of pathological findings, and the restrictions around sharing of patient data.

Original languageEnglish (US)
Title of host publicationSimulation and Synthesis in Medical Imaging - Third International Workshop, SASHIMI 2018, Held in Conjunction with MICCAI 2018, Proceedings
EditorsOrcun Goksel, Ipek Oguz, Ali Gooya, Ninon Burgos
PublisherSpringer Verlag
Pages1-11
Number of pages11
ISBN (Print)9783030005351
DOIs
StatePublished - Jan 1 2018
Event3rd International Workshop on Simulation and Synthesis in Medical Imaging, SASHIMI 2018 Held in Conjunction with 21st International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2018 - Granada, Spain
Duration: Sep 16 2018Sep 16 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11037 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference3rd International Workshop on Simulation and Synthesis in Medical Imaging, SASHIMI 2018 Held in Conjunction with 21st International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2018
CountrySpain
CityGranada
Period9/16/189/16/18

Fingerprint

Data Augmentation
Medical Image
Tumors
Medical imaging
Synthesis
Magnetic resonance imaging
Brain
Medical Imaging
Tumor
Segmentation
Learning systems
Brain Tumor
Generative Models
Synthetic Data
Demonstrate
Incidence
Machine Learning
Sharing
Restriction
Deep learning

Keywords

  • Brain tumor
  • Deep learning
  • GAN
  • Generative models
  • Image synthesis
  • Magnetic resonance imaging
  • MRI
  • Segmentation

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Shin, H. C., Tenenholtz, N. A., Rogers, J. K., Schwarz, C., Senjem, M. L., Gunter, J. L., ... Michalski, M. (2018). Medical image synthesis for data augmentation and anonymization using generative adversarial networks. In O. Goksel, I. Oguz, A. Gooya, & N. Burgos (Eds.), Simulation and Synthesis in Medical Imaging - Third International Workshop, SASHIMI 2018, Held in Conjunction with MICCAI 2018, Proceedings (pp. 1-11). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11037 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-00536-8_1

Medical image synthesis for data augmentation and anonymization using generative adversarial networks. / Shin, Hoo Chang; Tenenholtz, Neil A.; Rogers, Jameson K.; Schwarz, Christopher; Senjem, Matthew L.; Gunter, Jeffrey L.; Andriole, Katherine P.; Michalski, Mark.

Simulation and Synthesis in Medical Imaging - Third International Workshop, SASHIMI 2018, Held in Conjunction with MICCAI 2018, Proceedings. ed. / Orcun Goksel; Ipek Oguz; Ali Gooya; Ninon Burgos. Springer Verlag, 2018. p. 1-11 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11037 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Shin, HC, Tenenholtz, NA, Rogers, JK, Schwarz, C, Senjem, ML, Gunter, JL, Andriole, KP & Michalski, M 2018, Medical image synthesis for data augmentation and anonymization using generative adversarial networks. in O Goksel, I Oguz, A Gooya & N Burgos (eds), Simulation and Synthesis in Medical Imaging - Third International Workshop, SASHIMI 2018, Held in Conjunction with MICCAI 2018, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11037 LNCS, Springer Verlag, pp. 1-11, 3rd International Workshop on Simulation and Synthesis in Medical Imaging, SASHIMI 2018 Held in Conjunction with 21st International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2018, Granada, Spain, 9/16/18. https://doi.org/10.1007/978-3-030-00536-8_1
Shin HC, Tenenholtz NA, Rogers JK, Schwarz C, Senjem ML, Gunter JL et al. Medical image synthesis for data augmentation and anonymization using generative adversarial networks. In Goksel O, Oguz I, Gooya A, Burgos N, editors, Simulation and Synthesis in Medical Imaging - Third International Workshop, SASHIMI 2018, Held in Conjunction with MICCAI 2018, Proceedings. Springer Verlag. 2018. p. 1-11. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-030-00536-8_1
Shin, Hoo Chang ; Tenenholtz, Neil A. ; Rogers, Jameson K. ; Schwarz, Christopher ; Senjem, Matthew L. ; Gunter, Jeffrey L. ; Andriole, Katherine P. ; Michalski, Mark. / Medical image synthesis for data augmentation and anonymization using generative adversarial networks. Simulation and Synthesis in Medical Imaging - Third International Workshop, SASHIMI 2018, Held in Conjunction with MICCAI 2018, Proceedings. editor / Orcun Goksel ; Ipek Oguz ; Ali Gooya ; Ninon Burgos. Springer Verlag, 2018. pp. 1-11 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{a4fd095c0cf340378b415096426a2c45,
title = "Medical image synthesis for data augmentation and anonymization using generative adversarial networks",
abstract = "Data diversity is critical to success when training deep learning models. Medical imaging data sets are often imbalanced as pathologic findings are generally rare, which introduces significant challenges when training deep learning models. In this work, we propose a method to generate synthetic abnormal MRI images with brain tumors by training a generative adversarial network using two publicly available data sets of brain MRI. We demonstrate two unique benefits that the synthetic images provide. First, we illustrate improved performance on tumor segmentation by leveraging the synthetic images as a form of data augmentation. Second, we demonstrate the value of generative models as an anonymization tool, achieving comparable tumor segmentation results when trained on the synthetic data versus when trained on real subject data. Together, these results offer a potential solution to two of the largest challenges facing machine learning in medical imaging, namely the small incidence of pathological findings, and the restrictions around sharing of patient data.",
keywords = "Brain tumor, Deep learning, GAN, Generative models, Image synthesis, Magnetic resonance imaging, MRI, Segmentation",
author = "Shin, {Hoo Chang} and Tenenholtz, {Neil A.} and Rogers, {Jameson K.} and Christopher Schwarz and Senjem, {Matthew L.} and Gunter, {Jeffrey L.} and Andriole, {Katherine P.} and Mark Michalski",
year = "2018",
month = "1",
day = "1",
doi = "10.1007/978-3-030-00536-8_1",
language = "English (US)",
isbn = "9783030005351",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "1--11",
editor = "Orcun Goksel and Ipek Oguz and Ali Gooya and Ninon Burgos",
booktitle = "Simulation and Synthesis in Medical Imaging - Third International Workshop, SASHIMI 2018, Held in Conjunction with MICCAI 2018, Proceedings",

}

TY - GEN

T1 - Medical image synthesis for data augmentation and anonymization using generative adversarial networks

AU - Shin, Hoo Chang

AU - Tenenholtz, Neil A.

AU - Rogers, Jameson K.

AU - Schwarz, Christopher

AU - Senjem, Matthew L.

AU - Gunter, Jeffrey L.

AU - Andriole, Katherine P.

AU - Michalski, Mark

PY - 2018/1/1

Y1 - 2018/1/1

N2 - Data diversity is critical to success when training deep learning models. Medical imaging data sets are often imbalanced as pathologic findings are generally rare, which introduces significant challenges when training deep learning models. In this work, we propose a method to generate synthetic abnormal MRI images with brain tumors by training a generative adversarial network using two publicly available data sets of brain MRI. We demonstrate two unique benefits that the synthetic images provide. First, we illustrate improved performance on tumor segmentation by leveraging the synthetic images as a form of data augmentation. Second, we demonstrate the value of generative models as an anonymization tool, achieving comparable tumor segmentation results when trained on the synthetic data versus when trained on real subject data. Together, these results offer a potential solution to two of the largest challenges facing machine learning in medical imaging, namely the small incidence of pathological findings, and the restrictions around sharing of patient data.

AB - Data diversity is critical to success when training deep learning models. Medical imaging data sets are often imbalanced as pathologic findings are generally rare, which introduces significant challenges when training deep learning models. In this work, we propose a method to generate synthetic abnormal MRI images with brain tumors by training a generative adversarial network using two publicly available data sets of brain MRI. We demonstrate two unique benefits that the synthetic images provide. First, we illustrate improved performance on tumor segmentation by leveraging the synthetic images as a form of data augmentation. Second, we demonstrate the value of generative models as an anonymization tool, achieving comparable tumor segmentation results when trained on the synthetic data versus when trained on real subject data. Together, these results offer a potential solution to two of the largest challenges facing machine learning in medical imaging, namely the small incidence of pathological findings, and the restrictions around sharing of patient data.

KW - Brain tumor

KW - Deep learning

KW - GAN

KW - Generative models

KW - Image synthesis

KW - Magnetic resonance imaging

KW - MRI

KW - Segmentation

UR - http://www.scopus.com/inward/record.url?scp=85053935181&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85053935181&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-00536-8_1

DO - 10.1007/978-3-030-00536-8_1

M3 - Conference contribution

AN - SCOPUS:85053935181

SN - 9783030005351

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 1

EP - 11

BT - Simulation and Synthesis in Medical Imaging - Third International Workshop, SASHIMI 2018, Held in Conjunction with MICCAI 2018, Proceedings

A2 - Goksel, Orcun

A2 - Oguz, Ipek

A2 - Gooya, Ali

A2 - Burgos, Ninon

PB - Springer Verlag

ER -