A Comparison Study for Disaster Tweet Classification Using Deep Learning Models

Soudabeh Dinani, Doina Caragea

2023

Abstract

Effectively filtering and categorizing the large volume of user-generated content on social media during disaster events can help emergency management and disaster response prioritize their resources. Deep learning approaches, including recurrent neural networks and transformer-based models, have been previously used for this purpose. Capsule Neural Networks (CapsNets), initially proposed for image classification, have been proven to be useful for text analysis as well. However, to the best of our knowledge, CapsNets have not been used for classifying crisis-related messages, and have not been extensively compared with state-of-the-art transformer-based models, such as BERT. Therefore, in this study, we performed a thorough comparison between CapsNet models, state-of-the-art BERT models and two popular recurrent neural network models that have been successfully used for tweet classification, specifically, LSTM and Bi-LSTM models, on the task of classifying crisis tweets both in terms of their informativeness (binary classification), as well as their humanitarian content (multi-class classification). For this purpose, we used several benchmark datasets for crisis tweet classification, namely CrisisBench, CrisisNLP and CrisisLex. Experimental results show that the performance of the CapsNet models is on a par with that of LSTM and Bi-LSTM models for all metrics considered, while the performance obtained with BERT models have surpassed the performance of the other three models across different datasets and classes for both classification tasks, and thus BERT could be considered the best overall model for classifying crisis tweets.

Paper Citation

in Harvard Style

Dinani S. and Caragea D. (2023). A Comparison Study for Disaster Tweet Classification Using Deep Learning Models. In Proceedings of the 12th International Conference on Data Science, Technology and Applications - Volume 1: DATA; ISBN 978-989-758-664-4, SciTePress, pages 152-163. DOI: 10.5220/0012129300003541

in Bibtex Style

@conference{data23,
author={Soudabeh Dinani and Doina Caragea},
title={A Comparison Study for Disaster Tweet Classification Using Deep Learning Models},
booktitle={Proceedings of the 12th International Conference on Data Science, Technology and Applications - Volume 1: DATA},
year={2023},
pages={152-163},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012129300003541},
isbn={978-989-758-664-4},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 12th International Conference on Data Science, Technology and Applications - Volume 1: DATA
TI - A Comparison Study for Disaster Tweet Classification Using Deep Learning Models
SN - 978-989-758-664-4
AU - Dinani S.
AU - Caragea D.
PY - 2023
SP - 152
EP - 163
DO - 10.5220/0012129300003541
PB - SciTePress