Authors:
Mariana Fidalgo Fernandes
1
and
Plinio Moreno
2
Affiliations:
1
Instituto Superior Técnico, Universidade de Lisboa, 1049-001 Lisboa, Portugal
;
2
Institute for Systems and Robotics, Instituto Superior Técnico, Universidade de Lisboa, Torre Norte Piso 7, 1049-001 Lisboa, Portugal
Keyword(s):
Natural Language Processing, Deep Learning, Machine Translation, Transformer, Attention Mechanism.
Abstract:
Generative pre-trained transformers belong to the breakthroughs in Natural Language Processing (NLP), allowing Human-Robot Interactions (e.g. the creation of an open-domain chatbot). However, a substantial amount of research and available data are in English, causing low-resourced languages to be overlooked. This work addresses this problem for European Portuguese with two options: (i) Translation of the sentences before and after using the model fine-tuned on an English-based dataset, (ii) Translation of the English-based dataset to Portuguese and then fine-tune this model on it. We rely on the DialoGPT (dialogue generative pre-trained transformer), a tunable neural conversational answer generation model that learns the basic skills to conduct a dialogue. We use two sources of evaluation: (i) Metrics for text generation based on uncertainty (i.e. perplexity), and similarity between sentences (i.e. BLEU, METEOR and ROUGE) and (ii) Human-based evaluation of the sentences. The translat
ion of sentences before and after of the modified DialoGPT model, using the Daily Dialogue dataset led to the best results.
(More)