AI Models for Ultrasound Image Similarity Search: A Performance Evaluation

Petra Takacs, Richard Zsamboki, Elod Kiss, Ferdinand Dhombres

2025

Abstract

Querying similar images from a database to a reference image is an important task with multiple possible use-cases in healthcare industry, including improving labelling processes, and enhancing diagnostic support to medical professionals. The aim of this work is to measure the performance of different artificial neural networks, comparing their ability to identify clinically relevant similar images based on their generated feature sets. To measure the clinical relevance, metrics using expert labels of organs and diagnoses on the images were calculated, and image similarity was further confirmed by pixel metrics. Images with organ and diagnosis labels were selected from a dataset of early-stage pregnancy and 2nd -3rd trimester pregnancy ultrasound images respectively for the measurements. The networks were chosen from state-of-the-art foundational models trained on natural images, DINO and DINOv2, SAM2, and DreamSim. The best performing model based on our experiments is DreamSim for organ matches, and DINO for diagnosis matches. A simple ResNet trained on the mentioned early pregnancy dataset for organ classification was also added to the selection. ResNet performs best for early pregnancy organ matches, therefore finetuning a robust encoder on our own dataset is a promising future step to further enhance medically relevant similar image search.

Paper Citation

in Harvard Style

Takacs P., Zsamboki R., Kiss E. and Dhombres F. (2025). AI Models for Ultrasound Image Similarity Search: A Performance Evaluation. In Proceedings of the 18th International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 2: HEALTHINF; ISBN 978-989-758-731-3, SciTePress, pages 414-424. DOI: 10.5220/0013114700003911

in Bibtex Style

@conference{healthinf25,
author={Petra Takacs and Richard Zsamboki and Elod Kiss and Ferdinand Dhombres},
title={AI Models for Ultrasound Image Similarity Search: A Performance Evaluation},
booktitle={Proceedings of the 18th International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 2: HEALTHINF},
year={2025},
pages={414-424},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013114700003911},
isbn={978-989-758-731-3},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 18th International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 2: HEALTHINF
TI - AI Models for Ultrasound Image Similarity Search: A Performance Evaluation
SN - 978-989-758-731-3
AU - Takacs P.
AU - Zsamboki R.
AU - Kiss E.
AU - Dhombres F.
PY - 2025
SP - 414
EP - 424
DO - 10.5220/0013114700003911
PB - SciTePress