RIL-Contour: a Medical Imaging Dataset Annotation Tool for and with Deep Learning

Kenneth A. Philbrick, Alexander D. Weston, Zeynettin Akkus, Timothy Kline, Panagiotis Korfiatis, Tomas Sakinis, Petro Kostandy, Arunnit Boonrod, Atefeh Zeinoddini, Naoki M Takahashi, Bradley J Erickson

Research output: Contribution to journalArticle

Abstract

Deep-learning algorithms typically fall within the domain of supervised artificial intelligence and are designed to “learn” from annotated data. Deep-learning models require large, diverse training datasets for optimal model convergence. The effort to curate these datasets is widely regarded as a barrier to the development of deep-learning systems. We developed RIL-Contour to accelerate medical image annotation for and with deep-learning. A major goal driving the development of the software was to create an environment which enables clinically oriented users to utilize deep-learning models to rapidly annotate medical imaging. RIL-Contour supports using fully automated deep-learning methods, semi-automated methods, and manual methods to annotate medical imaging with voxel and/or text annotations. To reduce annotation error, RIL-Contour promotes the standardization of image annotations across a dataset. RIL-Contour accelerates medical imaging annotation through the process of annotation by iterative deep learning (AID). The underlying concept of AID is to iteratively annotate, train, and utilize deep-learning models during the process of dataset annotation and model development. To enable this, RIL-Contour supports workflows in which multiple-image analysts annotate medical images, radiologists approve the annotations, and data scientists utilize these annotations to train deep-learning models. To automate the feedback loop between data scientists and image analysts, RIL-Contour provides mechanisms to enable data scientists to push deep newly trained deep-learning models to other users of the software. RIL-Contour and the AID methodology accelerate dataset annotation and model development by facilitating rapid collaboration between analysts, radiologists, and engineers.

Original languageEnglish (US)
JournalJournal of Digital Imaging
DOIs
StatePublished - Jan 1 2019

Fingerprint

Medical imaging
Diagnostic Imaging
Learning
Software
Datasets
Deep learning
Workflow
Artificial Intelligence
Standardization
Learning algorithms
Artificial intelligence
Learning systems
Feedback
Engineers

Keywords

  • Annotation by iterative deep learing (AID)
  • Classification
  • Deep-learning
  • Medical image annotation
  • Segmentation
  • Software tools

ASJC Scopus subject areas

  • Radiological and Ultrasound Technology
  • Radiology Nuclear Medicine and imaging
  • Computer Science Applications

Cite this

RIL-Contour : a Medical Imaging Dataset Annotation Tool for and with Deep Learning. / Philbrick, Kenneth A.; Weston, Alexander D.; Akkus, Zeynettin; Kline, Timothy; Korfiatis, Panagiotis; Sakinis, Tomas; Kostandy, Petro; Boonrod, Arunnit; Zeinoddini, Atefeh; Takahashi, Naoki M; Erickson, Bradley J.

In: Journal of Digital Imaging, 01.01.2019.

Research output: Contribution to journalArticle

Philbrick, Kenneth A. ; Weston, Alexander D. ; Akkus, Zeynettin ; Kline, Timothy ; Korfiatis, Panagiotis ; Sakinis, Tomas ; Kostandy, Petro ; Boonrod, Arunnit ; Zeinoddini, Atefeh ; Takahashi, Naoki M ; Erickson, Bradley J. / RIL-Contour : a Medical Imaging Dataset Annotation Tool for and with Deep Learning. In: Journal of Digital Imaging. 2019.
@article{45ecc21b26b34c3da2125ce881dbf298,
title = "RIL-Contour: a Medical Imaging Dataset Annotation Tool for and with Deep Learning",
abstract = "Deep-learning algorithms typically fall within the domain of supervised artificial intelligence and are designed to “learn” from annotated data. Deep-learning models require large, diverse training datasets for optimal model convergence. The effort to curate these datasets is widely regarded as a barrier to the development of deep-learning systems. We developed RIL-Contour to accelerate medical image annotation for and with deep-learning. A major goal driving the development of the software was to create an environment which enables clinically oriented users to utilize deep-learning models to rapidly annotate medical imaging. RIL-Contour supports using fully automated deep-learning methods, semi-automated methods, and manual methods to annotate medical imaging with voxel and/or text annotations. To reduce annotation error, RIL-Contour promotes the standardization of image annotations across a dataset. RIL-Contour accelerates medical imaging annotation through the process of annotation by iterative deep learning (AID). The underlying concept of AID is to iteratively annotate, train, and utilize deep-learning models during the process of dataset annotation and model development. To enable this, RIL-Contour supports workflows in which multiple-image analysts annotate medical images, radiologists approve the annotations, and data scientists utilize these annotations to train deep-learning models. To automate the feedback loop between data scientists and image analysts, RIL-Contour provides mechanisms to enable data scientists to push deep newly trained deep-learning models to other users of the software. RIL-Contour and the AID methodology accelerate dataset annotation and model development by facilitating rapid collaboration between analysts, radiologists, and engineers.",
keywords = "Annotation by iterative deep learing (AID), Classification, Deep-learning, Medical image annotation, Segmentation, Software tools",
author = "Philbrick, {Kenneth A.} and Weston, {Alexander D.} and Zeynettin Akkus and Timothy Kline and Panagiotis Korfiatis and Tomas Sakinis and Petro Kostandy and Arunnit Boonrod and Atefeh Zeinoddini and Takahashi, {Naoki M} and Erickson, {Bradley J}",
year = "2019",
month = "1",
day = "1",
doi = "10.1007/s10278-019-00232-0",
language = "English (US)",
journal = "Journal of Digital Imaging",
issn = "0897-1889",
publisher = "Springer New York",

}

TY - JOUR

T1 - RIL-Contour

T2 - a Medical Imaging Dataset Annotation Tool for and with Deep Learning

AU - Philbrick, Kenneth A.

AU - Weston, Alexander D.

AU - Akkus, Zeynettin

AU - Kline, Timothy

AU - Korfiatis, Panagiotis

AU - Sakinis, Tomas

AU - Kostandy, Petro

AU - Boonrod, Arunnit

AU - Zeinoddini, Atefeh

AU - Takahashi, Naoki M

AU - Erickson, Bradley J

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Deep-learning algorithms typically fall within the domain of supervised artificial intelligence and are designed to “learn” from annotated data. Deep-learning models require large, diverse training datasets for optimal model convergence. The effort to curate these datasets is widely regarded as a barrier to the development of deep-learning systems. We developed RIL-Contour to accelerate medical image annotation for and with deep-learning. A major goal driving the development of the software was to create an environment which enables clinically oriented users to utilize deep-learning models to rapidly annotate medical imaging. RIL-Contour supports using fully automated deep-learning methods, semi-automated methods, and manual methods to annotate medical imaging with voxel and/or text annotations. To reduce annotation error, RIL-Contour promotes the standardization of image annotations across a dataset. RIL-Contour accelerates medical imaging annotation through the process of annotation by iterative deep learning (AID). The underlying concept of AID is to iteratively annotate, train, and utilize deep-learning models during the process of dataset annotation and model development. To enable this, RIL-Contour supports workflows in which multiple-image analysts annotate medical images, radiologists approve the annotations, and data scientists utilize these annotations to train deep-learning models. To automate the feedback loop between data scientists and image analysts, RIL-Contour provides mechanisms to enable data scientists to push deep newly trained deep-learning models to other users of the software. RIL-Contour and the AID methodology accelerate dataset annotation and model development by facilitating rapid collaboration between analysts, radiologists, and engineers.

AB - Deep-learning algorithms typically fall within the domain of supervised artificial intelligence and are designed to “learn” from annotated data. Deep-learning models require large, diverse training datasets for optimal model convergence. The effort to curate these datasets is widely regarded as a barrier to the development of deep-learning systems. We developed RIL-Contour to accelerate medical image annotation for and with deep-learning. A major goal driving the development of the software was to create an environment which enables clinically oriented users to utilize deep-learning models to rapidly annotate medical imaging. RIL-Contour supports using fully automated deep-learning methods, semi-automated methods, and manual methods to annotate medical imaging with voxel and/or text annotations. To reduce annotation error, RIL-Contour promotes the standardization of image annotations across a dataset. RIL-Contour accelerates medical imaging annotation through the process of annotation by iterative deep learning (AID). The underlying concept of AID is to iteratively annotate, train, and utilize deep-learning models during the process of dataset annotation and model development. To enable this, RIL-Contour supports workflows in which multiple-image analysts annotate medical images, radiologists approve the annotations, and data scientists utilize these annotations to train deep-learning models. To automate the feedback loop between data scientists and image analysts, RIL-Contour provides mechanisms to enable data scientists to push deep newly trained deep-learning models to other users of the software. RIL-Contour and the AID methodology accelerate dataset annotation and model development by facilitating rapid collaboration between analysts, radiologists, and engineers.

KW - Annotation by iterative deep learing (AID)

KW - Classification

KW - Deep-learning

KW - Medical image annotation

KW - Segmentation

KW - Software tools

UR - http://www.scopus.com/inward/record.url?scp=85066018289&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85066018289&partnerID=8YFLogxK

U2 - 10.1007/s10278-019-00232-0

DO - 10.1007/s10278-019-00232-0

M3 - Article

C2 - 31089974

AN - SCOPUS:85066018289

JO - Journal of Digital Imaging

JF - Journal of Digital Imaging

SN - 0897-1889

ER -