FormalStyler: GPT based Model for Formal Style Transfer based on Formality and Meaning Preservation

Mariano de Rivero, Cristhiam Tirado, Willy Ugarte

2021

Abstract

Style transfer is a natural language processing generation task, it consists of substituting one given writing style for another one. In this work, we seek to perform informal-to-formal style transfers in the English language. This process is shown in our web interface where the user input a informal message by text or voice. This project’s target audience are students and professionals in the need to improve the quality of their work by formalizing their texts. A style transfer is considered successful when the original semantic meaning of the message is preserved after the independent style has been replaced. This task is hindered by the scarcity of training and evaluation datasets alongside the lack of metrics. To accomplish this task we opted to utilize OpenAI’s GPT-2 Transformer-based pre-trained model. To adapt the GPT-2 to our research, we fine-tuned the model with a parallel corpus containing informal text entries paired with the equivalent formal ones. We evaluate the fine-tuned model results with two specific metrics, formality and meaning preservation. To further fine-tune the model we integrate a human-based feedback system where the user selects the best formal sentence out of the ones generated by the model. The resulting evaluations of our solution exhibit similar to improved scores in formality and meaning preservation to state-of-the-art approaches.

Paper Citation

in Harvard Style

de Rivero M., Tirado C. and Ugarte W. (2021). FormalStyler: GPT based Model for Formal Style Transfer based on Formality and Meaning Preservation. In Proceedings of the 13th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K 2021) - Volume 1: KDIR; ISBN 978-989-758-533-3, SciTePress, pages 48-56. DOI: 10.5220/0010674300003064

in Bibtex Style

@conference{kdir21,
author={Mariano de Rivero and Cristhiam Tirado and Willy Ugarte},
title={FormalStyler: GPT based Model for Formal Style Transfer based on Formality and Meaning Preservation},
booktitle={Proceedings of the 13th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K 2021) - Volume 1: KDIR},
year={2021},
pages={48-56},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010674300003064},
isbn={978-989-758-533-3},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 13th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K 2021) - Volume 1: KDIR
TI - FormalStyler: GPT based Model for Formal Style Transfer based on Formality and Meaning Preservation
SN - 978-989-758-533-3
AU - de Rivero M.
AU - Tirado C.
AU - Ugarte W.
PY - 2021
SP - 48
EP - 56
DO - 10.5220/0010674300003064
PB - SciTePress