Thyroid Classification in Ultraound by Deep Multimodal Learning

Jiong Shao, Hanshuo Xing, Mengying Li, Xinglong Wu

2022

Abstract

Purpose: Biopsy results are the gold standard for testing the benignity and malignancy of thyroid cancer, but they also brings the problems of overdiagnosis and overtreatment. This is a challenging task to avoid these two problems while ensuring diagnostic accuracy and efficiency. In this paper, we use deep learning multimodal models to assist physicians in diagnosis and improve diagnostic accuracy. Methods: This paper presents a multimodal deep learning model to assist physicians in the diagnosis of thyroid tumor. The model uses ultrasound images of the patient, geometric features of the lesion site, and clinical information to fuse modeling, with clinical information as the first modality, geometric features of the images as the second modality, and medical images as the third modality. The results are compared with other single-modal models to analyze and validate the performance of the multimodal model. Results: For the dataset used, the multimodal model had an accuracy of 0.884, precision of 0.865, recall of 0.859, and F1 of 0.862, the Area Under Curve (AUC) of the multimodal mode was 0.933, the AUC of the ResNet50 was 0.639, the AUC of the InceptionResnetV2 was 0.612, the AUC of the Densenet121 was 0.654, and the AUC of the EfficientNetB3 was 0.649. Conclusion: The multimodal model has high accuracy, sensitivity, and specificity in distinguishing benign and malignant thyroid tumors, and its performance is significantly better than the four single-modal deep learning classification models used for comparison. The proposed method is therefore valuable and is expected to help clinicians diagnose thyroid cancer efficiently.

Paper Citation

in Harvard Style

Shao J., Xing H., Li M. and Wu X. (2022). Thyroid Classification in Ultraound by Deep Multimodal Learning. In Proceedings of the 3rd International Symposium on Automation, Information and Computing - Volume 1: ISAIC; ISBN 978-989-758-622-4, SciTePress, pages 216-223. DOI: 10.5220/0011918100003612

in Bibtex Style

@conference{isaic22,
author={Jiong Shao and Hanshuo Xing and Mengying Li and Xinglong Wu},
title={Thyroid Classification in Ultraound by Deep Multimodal Learning},
booktitle={Proceedings of the 3rd International Symposium on Automation, Information and Computing - Volume 1: ISAIC},
year={2022},
pages={216-223},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011918100003612},
isbn={978-989-758-622-4},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 3rd International Symposium on Automation, Information and Computing - Volume 1: ISAIC
TI - Thyroid Classification in Ultraound by Deep Multimodal Learning
SN - 978-989-758-622-4
AU - Shao J.
AU - Xing H.
AU - Li M.
AU - Wu X.
PY - 2022
SP - 216
EP - 223
DO - 10.5220/0011918100003612
PB - SciTePress