NewsRecs: A Mobile App Framework for Conducting and Evaluating Online Experiments for News Recommender Systems

Noah Janzen, Fatih Gedikli

2023

Abstract

News recommender systems are ubiquitous on the web. Intensive research has been conducted over the last decades, resulting in the continuous proposal of new recommendation techniques based on Machine Learning models. To evaluate the performance of recommendation algorithms, offline experiments, user studies, and online experiments should ideally be carried out one after the other so that the candidates move through a quality funnel. However, our literature review of multiple academic papers shows that new models have generally been evaluated using offline experiments only. Presumably, this is because researchers rarely have access to a production system. This work attempts to alleviate this problem by presenting a framework that can be used to evaluate recommendation models for news articles in an online scenario. The framework consists of a mobile app in which users can receive recommendations from different algorithms depending on their assigned group and rate them in multiple ways. The backend collects log data and makes it available for the final evaluation. The specific contributions our article will make are as follows: (1) A thematic review of 27 academic experiments from the news recommendation domain focusing on the evaluation design. (2) An open-source mobile app framework for conducting and evaluating online experiments.

Paper Citation

in Harvard Style

Janzen N. and Gedikli F. (2023). NewsRecs: A Mobile App Framework for Conducting and Evaluating Online Experiments for News Recommender Systems. In Proceedings of the 15th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART, ISBN 978-989-758-623-1, pages 267-275. DOI: 10.5220/0011658000003393

in Bibtex Style

@conference{icaart23,
author={Noah Janzen and Fatih Gedikli},
title={NewsRecs: A Mobile App Framework for Conducting and Evaluating Online Experiments for News Recommender Systems},
booktitle={Proceedings of the 15th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART,},
year={2023},
pages={267-275},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011658000003393},
isbn={978-989-758-623-1},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 15th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART,
TI - NewsRecs: A Mobile App Framework for Conducting and Evaluating Online Experiments for News Recommender Systems
SN - 978-989-758-623-1
AU - Janzen N.
AU - Gedikli F.
PY - 2023
SP - 267
EP - 275
DO - 10.5220/0011658000003393