Attention! Transformer with Sentiment on Cryptocurrencies Price Prediction

Huali Zhao, Martin Crane, Marija Bezbradica

2022

Abstract

Cryptocurrencies have won a lot of attention as an investment tool in recent years. Specific research has been done on cryptocurrencies’ price prediction while the prices surge up. Classic models and recurrent neural networks are applied for the time series forecast. However, there remains limited research on how the Transformer works on forecasting cryptocurrencies price data. This paper investigated the forecasting capability of the Transformer model on Bitcoin (BTC) price data and Ethereum (ETH) price data which are time series with high fluctuation. Long short term memory model (LSTM) is employed for performance comparison. The result shows that LSTM performs better than Transformer both on BTC and ETH price prediction. Furthermore, in this paper, we also investigated if sentiment analysis can help improve the model’s performance in forecasting future prices. Twitter data and Valence Aware Dictionary and sEntiment Reasoner (VADER) is used for getting sentiment scores. The result shows that the sentiment analysis improves the Transformer model’s performance on BTC price but not ETH price. For the LSTM model, the sentiment analysis does not help with prediction results. Finally, this paper also shows that transfer learning can help on improving the Transformer’s prediction ability on ETH price data.

Paper Citation

in Harvard Style

Zhao H., Crane M. and Bezbradica M. (2022). Attention! Transformer with Sentiment on Cryptocurrencies Price Prediction. In Proceedings of the 7th International Conference on Complexity, Future Information Systems and Risk - Volume 1: COMPLEXIS, ISBN 978-989-758-565-4, pages 98-104. DOI: 10.5220/0011103400003197

in Bibtex Style

@conference{complexis22,
author={Huali Zhao and Martin Crane and Marija Bezbradica},
title={Attention! Transformer with Sentiment on Cryptocurrencies Price Prediction},
booktitle={Proceedings of the 7th International Conference on Complexity, Future Information Systems and Risk - Volume 1: COMPLEXIS,},
year={2022},
pages={98-104},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011103400003197},
isbn={978-989-758-565-4},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 7th International Conference on Complexity, Future Information Systems and Risk - Volume 1: COMPLEXIS,
TI - Attention! Transformer with Sentiment on Cryptocurrencies Price Prediction
SN - 978-989-758-565-4
AU - Zhao H.
AU - Crane M.
AU - Bezbradica M.
PY - 2022
SP - 98
EP - 104
DO - 10.5220/0011103400003197