Comparative Analysis of Neural Translation Models based on Transformers Architecture

Alexander Smirnov, Nikolay Teslya, a Shilov, Diethard Frank, Elena Minina, Martin Kovacs

2022

Abstract

While processing customers’ feedback for an industrial company, one of the important tasks is the classification of customer inquiries. However, this task can produce a number of difficulties when the text of the message can be composed using a large number of languages. One of the solutions, in this case, is to determine the language of the text and translate it into a base language, for which the classifier will be developed. This paper compares open models for automatic translation of texts. The following models based on the Transformers architecture were selected for comparison: M2M100, mBART, OPUS-MT (Helsinki NLP). A test data set was formed containing texts specific to the subject area. Microsoft Azure Translation was chosen as the reference translation. Translations produced by each model were compared with the reference translation using two metrics: BLEU and METEOR. The possibility of fast fine-tuning of models was also investigated to improve the quality of the translation of texts in the problem area. Among the reviewed models, M2M100 turned out to be the best in terms of translation quality, but it is also the most difficult to fine-tune it.

Paper Citation

in Harvard Style

Smirnov A., Teslya N., Shilov A., Frank D., Minina E. and Kovacs M. (2022). Comparative Analysis of Neural Translation Models based on Transformers Architecture. In Proceedings of the 24th International Conference on Enterprise Information Systems - Volume 2: ICEIS, ISBN 978-989-758-569-2, pages 586-593. DOI: 10.5220/0011083600003179

in Bibtex Style

@conference{iceis22,
author={Alexander Smirnov and Nikolay Teslya and a Shilov and Diethard Frank and Elena Minina and Martin Kovacs},
title={Comparative Analysis of Neural Translation Models based on Transformers Architecture},
booktitle={Proceedings of the 24th International Conference on Enterprise Information Systems - Volume 2: ICEIS,},
year={2022},
pages={586-593},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011083600003179},
isbn={978-989-758-569-2},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 24th International Conference on Enterprise Information Systems - Volume 2: ICEIS,
TI - Comparative Analysis of Neural Translation Models based on Transformers Architecture
SN - 978-989-758-569-2
AU - Smirnov A.
AU - Teslya N.
AU - Shilov A.
AU - Frank D.
AU - Minina E.
AU - Kovacs M.
PY - 2022
SP - 586
EP - 593
DO - 10.5220/0011083600003179