Spatial Organization and Molecular Correlation of Tumor-Infiltrating Lymphocytes Using Deep Learning on Pathology Images

The Cancer Genome Atlas Research Network

Research output: Contribution to journalArticle

45 Citations (Scopus)

Abstract

Beyond sample curation and basic pathologic characterization, the digitized H&E-stained images of TCGA samples remain underutilized. To highlight this resource, we present mappings of tumor-infiltrating lymphocytes (TILs) based on H&E images from 13 TCGA tumor types. These TIL maps are derived through computational staining using a convolutional neural network trained to classify patches of images. Affinity propagation revealed local spatial structure in TIL patterns and correlation with overall survival. TIL map structural patterns were grouped using standard histopathological parameters. These patterns are enriched in particular T cell subpopulations derived from molecular measures. TIL densities and spatial structure were differentially enriched among tumor types, immune subtypes, and tumor molecular subtypes, implying that spatial infiltrate state could reflect particular tumor cell aberration states. Obtaining spatial lymphocytic patterns linked to the rich genomic characterization of TCGA samples demonstrates one use for the TCGA image archives with insights into the tumor-immune microenvironment. Tumor-infiltrating lymphocytes (TILs) were identified from standard pathology cancer images by a deep-learning-derived “computational stain” developed by Saltz et al. They processed 5,202 digital images from 13 cancer types. Resulting TIL maps were correlated with TCGA molecular data, relating TIL content to survival, tumor subtypes, and immune profiles.

Original languageEnglish (US)
Pages (from-to)181-193.e7
JournalCell Reports
Volume23
Issue number1
DOIs
StatePublished - Apr 3 2018

Fingerprint

Tumor-Infiltrating Lymphocytes
Lymphocytes
Pathology
Tumors
Learning
Neoplasms
Tumor Microenvironment
Deep learning
Coloring Agents
Staining and Labeling
T-cells
T-Lymphocytes
Aberrations

Keywords

  • artificial intelligence
  • bioinformatics
  • computer vision
  • deep learning
  • digital pathology
  • immuno-oncology
  • lymphocytes
  • machine learning
  • tumor microenvironment
  • tumor-infiltrating lymphocytes

ASJC Scopus subject areas

  • Biochemistry, Genetics and Molecular Biology(all)

Cite this

Spatial Organization and Molecular Correlation of Tumor-Infiltrating Lymphocytes Using Deep Learning on Pathology Images. / The Cancer Genome Atlas Research Network.

In: Cell Reports, Vol. 23, No. 1, 03.04.2018, p. 181-193.e7.

Research output: Contribution to journalArticle

The Cancer Genome Atlas Research Network. / Spatial Organization and Molecular Correlation of Tumor-Infiltrating Lymphocytes Using Deep Learning on Pathology Images. In: Cell Reports. 2018 ; Vol. 23, No. 1. pp. 181-193.e7.
@article{b44d9b28b3904b119821f94538f5638e,
title = "Spatial Organization and Molecular Correlation of Tumor-Infiltrating Lymphocytes Using Deep Learning on Pathology Images",
abstract = "Beyond sample curation and basic pathologic characterization, the digitized H&E-stained images of TCGA samples remain underutilized. To highlight this resource, we present mappings of tumor-infiltrating lymphocytes (TILs) based on H&E images from 13 TCGA tumor types. These TIL maps are derived through computational staining using a convolutional neural network trained to classify patches of images. Affinity propagation revealed local spatial structure in TIL patterns and correlation with overall survival. TIL map structural patterns were grouped using standard histopathological parameters. These patterns are enriched in particular T cell subpopulations derived from molecular measures. TIL densities and spatial structure were differentially enriched among tumor types, immune subtypes, and tumor molecular subtypes, implying that spatial infiltrate state could reflect particular tumor cell aberration states. Obtaining spatial lymphocytic patterns linked to the rich genomic characterization of TCGA samples demonstrates one use for the TCGA image archives with insights into the tumor-immune microenvironment. Tumor-infiltrating lymphocytes (TILs) were identified from standard pathology cancer images by a deep-learning-derived “computational stain” developed by Saltz et al. They processed 5,202 digital images from 13 cancer types. Resulting TIL maps were correlated with TCGA molecular data, relating TIL content to survival, tumor subtypes, and immune profiles.",
keywords = "artificial intelligence, bioinformatics, computer vision, deep learning, digital pathology, immuno-oncology, lymphocytes, machine learning, tumor microenvironment, tumor-infiltrating lymphocytes",
author = "{The Cancer Genome Atlas Research Network} and Joel Saltz and Rajarsi Gupta and Le Hou and Tahsin Kurc and Pankaj Singh and Vu Nguyen and Dimitris Samaras and Shroyer, {Kenneth R.} and Tianhao Zhao and Rebecca Batiste and {Van Arnam}, John and Caesar-Johnson, {Samantha J.} and Demchok, {John A.} and Ina Felau and Melpomeni Kasapi and Ferguson, {Martin L.} and Hutter, {Carolyn M.} and Sofia, {Heidi J.} and Roy Tarnuzzer and Zhining Wang and Liming Yang and Zenklusen, {Jean C.} and Zhang, {Jiashan (Julia)} and Sudha Chudamani and Jia Liu and Laxmi Lolla and Rashi Naresh and Todd Pihl and Qiang Sun and Yunhu Wan and Ye Wu and Juok Cho and Timothy DeFreitas and Scott Frazer and Nils Gehlenborg and Gad Getz and Borad, {Mitesh J} and Vishal Chandan and John Cheville and Copland, {John A III} and Flotte, {Thomas J} and Michael Kendrick and Jean-Pierre Kocher and O'Neill, {Brian Patrick} and Patel, {Tushar C} and Petersen, {Gloria M} and Roberts, {Lewis Rowland} and Smallridge, {Robert Christian} and Melissa Stanton and Lizhi Zhang",
year = "2018",
month = "4",
day = "3",
doi = "10.1016/j.celrep.2018.03.086",
language = "English (US)",
volume = "23",
pages = "181--193.e7",
journal = "Cell Reports",
issn = "2211-1247",
publisher = "Cell Press",
number = "1",

}

TY - JOUR

T1 - Spatial Organization and Molecular Correlation of Tumor-Infiltrating Lymphocytes Using Deep Learning on Pathology Images

AU - The Cancer Genome Atlas Research Network

AU - Saltz, Joel

AU - Gupta, Rajarsi

AU - Hou, Le

AU - Kurc, Tahsin

AU - Singh, Pankaj

AU - Nguyen, Vu

AU - Samaras, Dimitris

AU - Shroyer, Kenneth R.

AU - Zhao, Tianhao

AU - Batiste, Rebecca

AU - Van Arnam, John

AU - Caesar-Johnson, Samantha J.

AU - Demchok, John A.

AU - Felau, Ina

AU - Kasapi, Melpomeni

AU - Ferguson, Martin L.

AU - Hutter, Carolyn M.

AU - Sofia, Heidi J.

AU - Tarnuzzer, Roy

AU - Wang, Zhining

AU - Yang, Liming

AU - Zenklusen, Jean C.

AU - Zhang, Jiashan (Julia)

AU - Chudamani, Sudha

AU - Liu, Jia

AU - Lolla, Laxmi

AU - Naresh, Rashi

AU - Pihl, Todd

AU - Sun, Qiang

AU - Wan, Yunhu

AU - Wu, Ye

AU - Cho, Juok

AU - DeFreitas, Timothy

AU - Frazer, Scott

AU - Gehlenborg, Nils

AU - Getz, Gad

AU - Borad, Mitesh J

AU - Chandan, Vishal

AU - Cheville, John

AU - Copland, John A III

AU - Flotte, Thomas J

AU - Kendrick, Michael

AU - Kocher, Jean-Pierre

AU - O'Neill, Brian Patrick

AU - Patel, Tushar C

AU - Petersen, Gloria M

AU - Roberts, Lewis Rowland

AU - Smallridge, Robert Christian

AU - Stanton, Melissa

AU - Zhang, Lizhi

PY - 2018/4/3

Y1 - 2018/4/3

N2 - Beyond sample curation and basic pathologic characterization, the digitized H&E-stained images of TCGA samples remain underutilized. To highlight this resource, we present mappings of tumor-infiltrating lymphocytes (TILs) based on H&E images from 13 TCGA tumor types. These TIL maps are derived through computational staining using a convolutional neural network trained to classify patches of images. Affinity propagation revealed local spatial structure in TIL patterns and correlation with overall survival. TIL map structural patterns were grouped using standard histopathological parameters. These patterns are enriched in particular T cell subpopulations derived from molecular measures. TIL densities and spatial structure were differentially enriched among tumor types, immune subtypes, and tumor molecular subtypes, implying that spatial infiltrate state could reflect particular tumor cell aberration states. Obtaining spatial lymphocytic patterns linked to the rich genomic characterization of TCGA samples demonstrates one use for the TCGA image archives with insights into the tumor-immune microenvironment. Tumor-infiltrating lymphocytes (TILs) were identified from standard pathology cancer images by a deep-learning-derived “computational stain” developed by Saltz et al. They processed 5,202 digital images from 13 cancer types. Resulting TIL maps were correlated with TCGA molecular data, relating TIL content to survival, tumor subtypes, and immune profiles.

AB - Beyond sample curation and basic pathologic characterization, the digitized H&E-stained images of TCGA samples remain underutilized. To highlight this resource, we present mappings of tumor-infiltrating lymphocytes (TILs) based on H&E images from 13 TCGA tumor types. These TIL maps are derived through computational staining using a convolutional neural network trained to classify patches of images. Affinity propagation revealed local spatial structure in TIL patterns and correlation with overall survival. TIL map structural patterns were grouped using standard histopathological parameters. These patterns are enriched in particular T cell subpopulations derived from molecular measures. TIL densities and spatial structure were differentially enriched among tumor types, immune subtypes, and tumor molecular subtypes, implying that spatial infiltrate state could reflect particular tumor cell aberration states. Obtaining spatial lymphocytic patterns linked to the rich genomic characterization of TCGA samples demonstrates one use for the TCGA image archives with insights into the tumor-immune microenvironment. Tumor-infiltrating lymphocytes (TILs) were identified from standard pathology cancer images by a deep-learning-derived “computational stain” developed by Saltz et al. They processed 5,202 digital images from 13 cancer types. Resulting TIL maps were correlated with TCGA molecular data, relating TIL content to survival, tumor subtypes, and immune profiles.

KW - artificial intelligence

KW - bioinformatics

KW - computer vision

KW - deep learning

KW - digital pathology

KW - immuno-oncology

KW - lymphocytes

KW - machine learning

KW - tumor microenvironment

KW - tumor-infiltrating lymphocytes

UR - http://www.scopus.com/inward/record.url?scp=85044586207&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85044586207&partnerID=8YFLogxK

U2 - 10.1016/j.celrep.2018.03.086

DO - 10.1016/j.celrep.2018.03.086

M3 - Article

VL - 23

SP - 181-193.e7

JO - Cell Reports

JF - Cell Reports

SN - 2211-1247

IS - 1

ER -