Language Identification of Similar Languages using Recurrent Neural Networks

Ermelinda Oro, Massimo Ruffolo, Mostafa Sheikhalishahi

2018

Abstract

The goal of similar Language IDentification (LID) is to quickly and accurately identify the language of the text. It plays an important role in several Natural Language Processing (NLP) applications where it is frequently used as a pre-processing technique. For example, information retrieval systems use LID as a filtering technique to provide users with documents written only in a given language. Although different approaches to this problem have been proposed, similar language identification, in particular applied to short texts, remains a challenging task in NLP. In this paper, a method that combines word vectors representation and Long Short-Term Memory (LSTM) has been implemented. The experimental evaluation on public and well-known datasets has shown that the proposed method improves accuracy and precision of language identification tasks.

Paper Citation

in Harvard Style

Oro E., Ruffolo M. and Sheikhalishahi M. (2018). Language Identification of Similar Languages using Recurrent Neural Networks.In Proceedings of the 10th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, ISBN 978-989-758-275-2, pages 635-640. DOI: 10.5220/0006678606350640

in Bibtex Style

@conference{icaart18,
author={Ermelinda Oro and Massimo Ruffolo and Mostafa Sheikhalishahi},
title={Language Identification of Similar Languages using Recurrent Neural Networks},
booktitle={Proceedings of the 10th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},
year={2018},
pages={635-640},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006678606350640},
isbn={978-989-758-275-2},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 10th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,
TI - Language Identification of Similar Languages using Recurrent Neural Networks
SN - 978-989-758-275-2
AU - Oro E.
AU - Ruffolo M.
AU - Sheikhalishahi M.
PY - 2018
SP - 635
EP - 640
DO - 10.5220/0006678606350640