Language Identification for Short Medical Texts

Erick Velazquez Godinez, Zoltán Szlávik, Selene Baez Santamaría, Robert-Jan Sips

2020

Abstract

Language identification remains a challenge for short texts originating from social media. Moreover, domain-specific terminology, which is frequent in the medical domain, may not change cross-linguistically, making language identification even more difficult. We conducted language identification on four datasets, two of them with general language, and two of them containing medical language. We evaluated the impact of two embedding representations and a set of linguistic features based on graphotactics. The proposed linguistic features reflect the graphotactics of the languages included in the test dataset. For classification, we implemented two algorithms: random forest and SVM. Our findings show that, when classifying general language, linguistic-based features perform close to the embedding representations of fastText and BERT. However, when classifying text with technical terms, the linguistic features outperform embedding representations. The combination of embeddings with linguistic features had a positive impact on the classification task under both settings. Therefore, our results suggest that these linguistic features could be applied for big and small datasets keeping the good performances in both general and medical languages. As future work, we want to test the linguistic features for a more significant set of languages.

Paper Citation

in Harvard Style

Godinez E., Szlávik Z., Santamaría S. and Sips R. (2020). Language Identification for Short Medical Texts. In Proceedings of the 13th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2020) - Volume 5: HEALTHINF; ISBN 978-989-758-398-8, SciTePress, pages 399-406. DOI: 10.5220/0008950903990406

in Bibtex Style

@conference{healthinf20,
author={Erick Velazquez Godinez and Zoltán Szlávik and Selene Baez Santamaría and Robert-Jan Sips},
title={Language Identification for Short Medical Texts},
booktitle={Proceedings of the 13th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2020) - Volume 5: HEALTHINF},
year={2020},
pages={399-406},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0008950903990406},
isbn={978-989-758-398-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 13th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2020) - Volume 5: HEALTHINF
TI - Language Identification for Short Medical Texts
SN - 978-989-758-398-8
AU - Godinez E.
AU - Szlávik Z.
AU - Santamaría S.
AU - Sips R.
PY - 2020
SP - 399
EP - 406
DO - 10.5220/0008950903990406
PB - SciTePress