An Off-line Evaluation of Users’ Ranking Metrics in Group Recommendation

Silvia Rossi, Francesco Cervone, Francesco Barile

2017

Abstract

One of the major issue in designing group recommendation techniques relates to the difficulty of the evaluation process. Up-today, no freely available dataset exists that contains information about groups, like, for example, the group’s choices or social aspects that may characterize the group’s members. The objective of the paper is to analyze the possibility to make an evaluation of ranking-based groups recommendation techniques by using offline testing. Typically, the evaluation of group recommendations is computed, as in the classical single user case, by comparing the predicted group’s ratings with respect to the single users’ ratings. Since the information contained in the datasets are mainly such user’s ratings, here, ratings are used to define different ranking metrics. Results suggest that such an attempt is hardly feasible. Performance seems not to be affected by the choice of ranking technique, except for some particular cases. This could be due to the averaging effect of the evaluation with respect to the single users’ ratings, so a deeper analysis or specific dataset are necessary.

References

  1. Amer-Yahia, S., Roy, S. B., Chawlat, A., Das, G., and Yu, C. (2009). Group recommendation: Semantics and efficiency. Proc. VLDB Endow., 2(1):754-765.
  2. Ardissono, L., Goy, A., Petrone, G., Segnan, M., and Torasso, P. (2003). Intrigue: Personalized recommendation of tourist attractions for desktop and handset devices. Applied Artificial Intelligence , 17(8):687-714.
  3. Baltrunas, L., Makcinskas, T., and Ricci, F. (2010). Group recommendations with rank aggregation and collaborative filtering. In Proc. of the Fourth ACM RecSys 7810, pages 119-126. ACM.
  4. Berkovsky, S. and Freyne, J. (2010). Group-based recipe recommendations: Analysis of data aggregation strategies. In Proc. of the Fourth ACM RecSys 7810, pages 111-118. ACM.
  5. Dooms, S., De Pessemier, T., and Martens, L. (2013). Movietweetings: a movie rating dataset collected from twitter. In Workshop on Crowdsourcing and human computation for recommender systems, volume 2013, page 43.
  6. Gartrell, M., Xing, X., Lv, Q., Beach, A., Han, R., Mishra, S., and Seada, K. (2010). Enhancing group recommendation by incorporating social relationship interactions. In Proceedings of the 16th ACM International Conference on Supporting Group Work, GROUP 7810, pages 97-106. ACM.
  7. Levine, J. M. and Moreland, R. L. (2008). Small groups: key readings. Psychology Press.
  8. Masthoff, J. (2011). Recommender Systems Handbook, chapter Group Recommender Systems: Combining Individual Models, pages 677-702. Springer US, Boston, MA.
  9. O'Connor, M., Cosley, D., Konstan, J. A., and Riedl, J. (2001). Polylens: A recommender system for groups of users. In Proc. of the 7th European Conf. on CSCW, pages 199-218.
  10. Pera, M. S. and Ng, Y.-K. (2013). A group recommender for movies based on content similarity and popularity. Inf. Process. Manage., 49(3):673-687.
  11. Rossi, S., Barile, F., Caso, A., and Rossi, A. (2016). Web Information Systems and Technologies: 11th International Conference, WEBIST, Revised Selected Papers, volume 246, chapter Pre-trip Ratings and Social Networks User Behaviors for Recommendations in Touristic Web Portals, pages 297-317. Springer International Publishing.
  12. Rossi, S., Barile, F., Di Martino, S., and Improta, D. (2017). A comparison of two preference elicitation approaches for museum recommendations. Concurrency and Computation: Practice and Experience, to appear.
  13. Rossi, S., Caso, A., and Barile, F. (2015). Combining users and items rankings for group decision support. Advances in Intelligent Systems and Computing, 372:151- 158.
  14. Rossi, S. and Cervone, F. (2016). Social utilities and personality traits for group recommendation: A pilot user study. In Proceedings of the 8th International Conference on Agents and Artificial Intelligence , pages 38- 46.
Download


Paper Citation


in Harvard Style

Rossi S., Cervone F. and Barile F. (2017). An Off-line Evaluation of Users’ Ranking Metrics in Group Recommendation . In Proceedings of the 9th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART, ISBN 978-989-758-219-6, pages 252-259. DOI: 10.5220/0006200702520259


in Bibtex Style

@conference{icaart17,
author={Silvia Rossi and Francesco Cervone and Francesco Barile},
title={An Off-line Evaluation of Users’ Ranking Metrics in Group Recommendation},
booktitle={Proceedings of the 9th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,},
year={2017},
pages={252-259},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006200702520259},
isbn={978-989-758-219-6},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 9th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,
TI - An Off-line Evaluation of Users’ Ranking Metrics in Group Recommendation
SN - 978-989-758-219-6
AU - Rossi S.
AU - Cervone F.
AU - Barile F.
PY - 2017
SP - 252
EP - 259
DO - 10.5220/0006200702520259