Truth Assessment of Objective Facts Extracted from Tweets: A Case Study on World Cup 2014 Game Facts

Bas Janssen, Mena Habib, Maurice van Keulen

Abstract

By understanding the tremendous opportunities to work with social media data and the acknowledgment of the negative effects social media messages can have, a way of assessing truth in claims on social media would not only be interesting but also very valuable. By making use of this ability, applications using social media data could be supported, or a selection tool in research regarding the spread of false rumors or ’fake news’ could be build. In this paper, we show that we can determine truth by using a statistical classifier supported by an architecture of three preprocessing phases. We base our research on a dataset of Twitter messages about the FIFA World Cup 2014. We determine the truth of a tweet by using 7 popular fact types (involving events in the matches in the tournament such as scoring a goal) and we show that we can achieve an F1-score of 0.988 for the first class; the Tweets which contain no false facts and an F1-score of 0.818 on the second class; the Tweets which contain one or more false facts.

References

  1. Bram Koster, M. (2014). Journalisten: social media niet betrouwbaar, wel belangrijk #sming14.
  2. Cano, A. E., Preotiuc-Pietro, D., Radovanovic, D., Weller, K., and Dadzie, A.-S. (2016). # microposts2016: 6th workshop on making sense of microposts: Big things come in small packages.
  3. Castillo, C., Mendoza, M., and Poblete, B. (2011). Information credibility on twitter.
  4. Declerck, T. and Lendvai, P. (2015). Processing and normalizing hashtags. Proc. of RANLP 2015.
  5. Derczynski, L., Strötgen, J., Maynard, D., Greenwood, M. A., and Jung, M. (2016). Gate-time: Extraction of temporal expressions and events.
  6. Gupta, A., Kumaraguru, P., Castillo, C., and Meier, P. (2014). Tweetcred: Real-time credibility assessment of content on twitter.
  7. Hamidian, S. and Diab, M. T. (2016). Rumor identification and belief investigation on twitter.
  8. Inc., N. H. (2012). State of the media - the social medai report 2012.
  9. Janssen, B. (2016). Determining truth in tweets using feature based supervised statistical classifiers. Master's thesis, University of Twente.
  10. Kang, M. (2010). Measuring social media credibility: A study on a measure of blog credibility. Institute for Public Relations, pages 59-68.
  11. Reuters Institute for the Study of Journalism (2013). Reuters institute digital news report 2013.
  12. Tech Crunch (2016). Facebook chose to fight fake news with ai, not just user reports.
  13. Twitter (2015). Insights into the WorldCup conversation on Twitter.
  14. Vakulenko, S., Göbel, M., Scharl, A., and Nixon, L. (2016). Visualising the propagation of news on the web.
  15. Zhao, Z., Resnick, P., and Mei, Q. (2015). Enquiring minds: Early detection of rumors in social media from enquiry posts.
  16. Zubiaga, A., Liakata, M., Procter, R., Hoi, G. W. S., and Tolmie, P. (2016). Analysing how people orient to and spread rumours in social media by looking at conversational threads. PloS one, 11(3):e0150989.
Download


Paper Citation


in Harvard Style

Janssen B., Habib M. and van Keulen M. (2017). Truth Assessment of Objective Facts Extracted from Tweets: A Case Study on World Cup 2014 Game Facts . In Proceedings of the 13th International Conference on Web Information Systems and Technologies - Volume 1: WEBIST, ISBN 978-989-758-246-2, pages 187-195. DOI: 10.5220/0006185101870195


in Bibtex Style

@conference{webist17,
author={Bas Janssen and Mena Habib and Maurice van Keulen},
title={Truth Assessment of Objective Facts Extracted from Tweets: A Case Study on World Cup 2014 Game Facts},
booktitle={Proceedings of the 13th International Conference on Web Information Systems and Technologies - Volume 1: WEBIST,},
year={2017},
pages={187-195},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006185101870195},
isbn={978-989-758-246-2},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 13th International Conference on Web Information Systems and Technologies - Volume 1: WEBIST,
TI - Truth Assessment of Objective Facts Extracted from Tweets: A Case Study on World Cup 2014 Game Facts
SN - 978-989-758-246-2
AU - Janssen B.
AU - Habib M.
AU - van Keulen M.
PY - 2017
SP - 187
EP - 195
DO - 10.5220/0006185101870195