On Enhancing Code-Mixed Sentiment and Emotion Classification Using FNet and FastFormer

Anuj Kumar, Amit Pandey, Satyadev Ahlawat, Yamuna Prasad

2025

Abstract

Code-mixing, the blending of multiple languages within a communication, is becoming increasingly common on social media. If left unchecked for sentiment analysis, this trend can lead to hate speech or violence, emphasizing the need for advanced techniques to interpret emotions and sentiments in code-mixed languages accurately. Current research has mainly focused on code-mixed text involving a limited number of languages. However, these methods often yield suboptimal results due to inadequate feature extraction by existing learning models. Additionally, achieving high accuracy and extracting meaningful features from code-mixed text remains a significant challenge. To address this, we propose two transformer-based feature extraction methods for sentiment and emotion classification in code-mixed text. The first method integrates the Fourier transform into the transformer-based cross-lingual language model, XLM-Roberta, by incorporating the encoder layers of Fourier Net (FNet). This Fourier encoder layer applies a Fourier transform to the final output vector of hidden states, enabling the model to capture complex patterns more effectively. The second method incorporates the encoding layers of FastFormer into the XLM-Roberta framework. FastFormer generates contextual embeddings using additive attention mechanisms, allowing for extracting more effective contextual features. Experimental results show that the proposed approaches improve accuracy compared to the state-of-the-art by 1.5% and 0.9% in sentiment detection and 3.9% and 1.97% in emotion detection on the publicly available SentiMix code-mixed benchmark dataset.

Paper Citation

in Harvard Style

Kumar A., Pandey A., Ahlawat S. and Prasad Y. (2025). On Enhancing Code-Mixed Sentiment and Emotion Classification Using FNet and FastFormer. In Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART; ISBN 978-989-758-737-5, SciTePress, pages 670-678. DOI: 10.5220/0013173600003890

in Bibtex Style

@conference{icaart25,
author={Anuj Kumar and Amit Pandey and Satyadev Ahlawat and Yamuna Prasad},
title={On Enhancing Code-Mixed Sentiment and Emotion Classification Using FNet and FastFormer},
booktitle={Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART},
year={2025},
pages={670-678},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013173600003890},
isbn={978-989-758-737-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART
TI - On Enhancing Code-Mixed Sentiment and Emotion Classification Using FNet and FastFormer
SN - 978-989-758-737-5
AU - Kumar A.
AU - Pandey A.
AU - Ahlawat S.
AU - Prasad Y.
PY - 2025
SP - 670
EP - 678
DO - 10.5220/0013173600003890
PB - SciTePress