TY - GEN
T1 - Pay Attention with Focus
T2 - 24th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2021
AU - Kalra, Shivam
AU - Adnan, Mohammed
AU - Hemati, Sobhan
AU - Dehkharghanian, Taher
AU - Rahnamayan, Shahryar
AU - Tizhoosh, Hamid R.
N1 - Publisher Copyright:
© 2021, Springer Nature Switzerland AG.
PY - 2021
Y1 - 2021
N2 - Deep learning methods such as convolutional neural networks (CNNs) are difficult to directly utilize to analyze whole slide images (WSIs) due to the large image dimensions. We overcome this limitation by proposing a novel two-stage approach. First, we extract a set of representative patches (called mosaic) from a WSI. Each patch of a mosaic is encoded to a feature vector using a deep network. The feature extractor model is fine-tuned using hierarchical target labels of WSIs, i.e., anatomic site and primary diagnosis. In the second stage, a set of encoded patch-level features from a WSI is used to compute the primary diagnosis probability through the proposed Pay Attention with Focus scheme, an attention-weighted averaging of predicted probabilities for all patches of a mosaic modulated by a trainable focal factor. Experimental results show that the proposed model can be robust, and effective for the classification of WSIs.
AB - Deep learning methods such as convolutional neural networks (CNNs) are difficult to directly utilize to analyze whole slide images (WSIs) due to the large image dimensions. We overcome this limitation by proposing a novel two-stage approach. First, we extract a set of representative patches (called mosaic) from a WSI. Each patch of a mosaic is encoded to a feature vector using a deep network. The feature extractor model is fine-tuned using hierarchical target labels of WSIs, i.e., anatomic site and primary diagnosis. In the second stage, a set of encoded patch-level features from a WSI is used to compute the primary diagnosis probability through the proposed Pay Attention with Focus scheme, an attention-weighted averaging of predicted probabilities for all patches of a mosaic modulated by a trainable focal factor. Experimental results show that the proposed model can be robust, and effective for the classification of WSIs.
KW - Classification
KW - Multi-instance learning
KW - Whole slide image
UR - http://www.scopus.com/inward/record.url?scp=85116421812&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85116421812&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-87237-3_34
DO - 10.1007/978-3-030-87237-3_34
M3 - Conference contribution
AN - SCOPUS:85116421812
SN - 9783030872366
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 350
EP - 359
BT - Medical Image Computing and Computer Assisted Intervention – MICCAI 2021 - 24th International Conference, Proceedings
A2 - de Bruijne, Marleen
A2 - Cattin, Philippe C.
A2 - Cotin, Stéphane
A2 - Padoy, Nicolas
A2 - Speidel, Stefanie
A2 - Zheng, Yefeng
A2 - Essert, Caroline
PB - Springer Science and Business Media Deutschland GmbH
Y2 - 27 September 2021 through 1 October 2021
ER -