Learning Similarity via Subjective Evaluations and Deep Features of Histopathology Images

Visual similarity estimation for histopathology images plays a key role in many medical imaging tasks, especially in image search and retrieval. All image similarity evaluation approaches employ distance-based metrics to quantify the degree of (dis) similarity. However, it has always been challenging to numerically estimate the similarity between two images, which is compatible with subjective assessment of the human operator, i.e., physicians such as radiologists and pathologists. Relying only on distance calculations through Euclidean, Manhattan, Hamming, and cosine distances does not provide us with the result that can be translated to human judgment in linguistic terms and/or in a normalized range. There is a need for a reliable image similarity measurement compatible with the human assessment with minimum possible conflict. This work proposes a new scheme that evaluates the similarity between a pair of histopathology images close to human reasoning using a fuzzy-logic approach. To this end, we developed a web application to interface with users and to collect descriptive image similarity data for training and testing purposes. We designed an adaptive neuro-fuzzy inference system (ANFIS) to model the vague and uncertain nature of user image assessment for the histopathology image comparison task. The experimental results show that the trained ANFIS can estimate the image similarity with acceptable accuracy and consistent with the user evaluations.