TY - GEN
T1 - Retrieval evaluation and distance learning from perceived similarity between endomicroscopy videos
AU - André, Barbara
AU - Vercauteren, Tom
AU - Buchner, Anna M.
AU - Wallace, Michael B.
AU - Ayache, Nicholas
PY - 2011
Y1 - 2011
N2 - Evaluating content-based retrieval (CBR) is challenging because it requires an adequate ground-truth. When the available ground-truth is limited to textual metadata such as pathological classes, retrieval results can only be evaluated indirectly, for example in terms of classification performance. In this study we first present a tool to generate perceived similarity ground-truth that enables direct evaluation of endomicroscopic video retrieval. This tool uses a four-points Likert scale and collects subjective pairwise similarities perceived by multiple expert observers. We then evaluate against the generated ground-truth a previously developed dense bag-of-visual-words method for endomicroscopic video retrieval. Confirming the results of previous indirect evaluation based on classification, our direct evaluation shows that this method significantly outperforms several other state-of-the-art CBR methods. In a second step, we propose to improve the CBR method by learning an adjusted similarity metric from the perceived similarity ground-truth. By minimizing a margin-based cost function that differentiates similar and dissimilar video pairs, we learn a weight vector applied to the visual word signatures of videos. Using cross-validation, we demonstrate that the learned similarity distance is significantly better correlated with the perceived similarity than the original visual-word-based distance.
AB - Evaluating content-based retrieval (CBR) is challenging because it requires an adequate ground-truth. When the available ground-truth is limited to textual metadata such as pathological classes, retrieval results can only be evaluated indirectly, for example in terms of classification performance. In this study we first present a tool to generate perceived similarity ground-truth that enables direct evaluation of endomicroscopic video retrieval. This tool uses a four-points Likert scale and collects subjective pairwise similarities perceived by multiple expert observers. We then evaluate against the generated ground-truth a previously developed dense bag-of-visual-words method for endomicroscopic video retrieval. Confirming the results of previous indirect evaluation based on classification, our direct evaluation shows that this method significantly outperforms several other state-of-the-art CBR methods. In a second step, we propose to improve the CBR method by learning an adjusted similarity metric from the perceived similarity ground-truth. By minimizing a margin-based cost function that differentiates similar and dissimilar video pairs, we learn a weight vector applied to the visual word signatures of videos. Using cross-validation, we demonstrate that the learned similarity distance is significantly better correlated with the perceived similarity than the original visual-word-based distance.
UR - http://www.scopus.com/inward/record.url?scp=82255164550&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=82255164550&partnerID=8YFLogxK
U2 - 10.1007/978-3-642-23626-6_37
DO - 10.1007/978-3-642-23626-6_37
M3 - Conference contribution
C2 - 22003712
AN - SCOPUS:82255164550
SN - 9783642236259
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 297
EP - 304
BT - Medical Image Computing and Computer-Assisted Intervention, MICCAI 2011 - 14th International Conference, Proceedings
T2 - 14th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2011
Y2 - 18 September 2011 through 22 September 2011
ER -