Knowledge-based Reliability Metrics for Social Media Accounts

Nuno Guimaraes, Alvaro Figueira, Luis Torgo

2020

Abstract

The growth of social media as an information medium without restrictive measures on the creation of new accounts led to the rise of malicious agents with the intend to diffuse unreliable information in the network, ultimately affecting the perception of users in important topics such as political and health issues. Although the problem is being tackled within the domain of bot detection, the impact of studies in this area is still limited due to 1) not all accounts that spread unreliable content are bots, 2) human-operated accounts are also responsible for the diffusion of unreliable information and 3) bot accounts are not always malicious (e.g. news aggregators). Also, most of these methods are based on supervised models that required annotated data and updates to maintain their performance through time. In this work, we build a framework and develop knowledge-based metrics to complement the current research in bot detection and characterize the impact and behavior of a Twitter account, independently of the way it is operated (human or bot). We proceed to analyze a sample of the accounts using the metrics proposed and evaluate the necessity of these metrics by comparing them with the scores from a bot detection system. The results show that the metrics can characterize different degrees of unreliable accounts, from unreliable bot accounts with a high number of followers to human-operated accounts that also spread unreliable content (but with less impact on the network). Furthermore, evaluating a sample of the accounts with a bot detection system shown that bots compose around 11% of the sample of unreliable accounts extracted and that the bot score is not correlated with the proposed metrics. In addition, the accounts that achieve the highest values in our metrics present different characteristics than the ones that achieve the highest bot score. This provides evidence on the usefulness of our metrics in the evaluation of unreliable accounts in social networks.

Paper Citation

in Harvard Style

Guimaraes N., Figueira A. and Torgo L. (2020). Knowledge-based Reliability Metrics for Social Media Accounts.In Proceedings of the 16th International Conference on Web Information Systems and Technologies - Volume 1: WEBIST, ISBN 978-989-758-478-7, pages 339-350. DOI: 10.5220/0010140403390350

in Bibtex Style

@conference{webist20,
author={Nuno Guimaraes and Alvaro Figueira and Luis Torgo},
title={Knowledge-based Reliability Metrics for Social Media Accounts},
booktitle={Proceedings of the 16th International Conference on Web Information Systems and Technologies - Volume 1: WEBIST,},
year={2020},
pages={339-350},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010140403390350},
isbn={978-989-758-478-7},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Conference on Web Information Systems and Technologies - Volume 1: WEBIST,
TI - Knowledge-based Reliability Metrics for Social Media Accounts
SN - 978-989-758-478-7
AU - Guimaraes N.
AU - Figueira A.
AU - Torgo L.
PY - 2020
SP - 339
EP - 350
DO - 10.5220/0010140403390350