EVALUATION OF RECOMMENDER SYSTEMS THROUGH SIMULATED USERS

Miquel Montaner, Beatriz López, Josep Lluís de la Rosa

2004

Abstract

Recommender systems have proved really useful in order to handle with the information overload on the Internet. However, it is very difficult to evaluate such a personalised systems since this involves purely subjective assessments. Actually, only very few recommender systems developed over the Internet evaluate and discuss their results scientifically. The contribution of this paper is a methodology for evaluating recommender systems: the ”profile discovering procedure”. Based on a list of item evaluations previously provided by a real user, this methodology simulates the recommendation process of a recommender system over time. Besides, an extension of this methodology has been designed in order to simulate the collaboration among users. At the end of the simulations, the desired evaluation measures (precision and recall among others) are presented. This methodology and its extensions have been successfully used in the evaluation of different parameters and algorithms of a restaurant recommender system.

References

  1. Amazon (2003). http://www.amazon.com.
  2. Berney, B. and Ferneley, E. (1999). CASMIR: Information retrieval based on collaborative user proling. In Proceedings of PAAM'99, pp. 41-56.
  3. Good, N., Schafer, J., Konstan, et al. (1999). Combining collaborative ltering with personal agents for better recommendations. In Proceedings of AAAI, vol 35, pp. 439-446. AAAI Press.
  4. Holte, R. C. and Yan, N. Y. (1996). Inferring what a user is not interested in. AAAI Spring Symp. on ML in Information Access. Stanford.
  5. Martnez, J. (2003). Agent Based Tool to Support the Con guration of Work Teams. PhD Thesis Project, UPC.
  6. Mladenic, D. (1996). Personal WebWatcher: Implementation and design. TR IJS-DP-7472, Department of Intelligent Systems, J.Stefan Institute, Slovenia.
  7. Montaner, M. (2003). Collaborative recommender agents based on CBR and trust. PhD Thesis in Computer Engineering. Universitat de Girona.
  8. Montaner, M., López, B., and de la Rosa, J. L. (2002). Opinion-based ltering through trust. In Proceedings of CIA'02. LNAI 2446., pp. 164-178.
  9. Montaner, M., López, B., and de la Rosa, J. L. (2003). A taxonomy of recommender agents on the Internet. Arti cial Intelligence Review, volume 19:4, pp. 285-330. Kluwer Academic Publishers.
  10. Sanguesa, R., Cortés, U., and Faltings, B. (2000). W9. Workshop on recommender systems. In Autonomous Agents, 2000. Barcelona, Spain.
  11. Sheth, B. (1994). A learning approach to personalized information ltering. M.S. Thesis, Massachusetts Institute of Technology.
Download


Paper Citation


in Harvard Style

Montaner M., López B. and Lluís de la Rosa J. (2004). EVALUATION OF RECOMMENDER SYSTEMS THROUGH SIMULATED USERS . In Proceedings of the Sixth International Conference on Enterprise Information Systems - Volume 3: ICEIS, ISBN 972-8865-00-7, pages 303-308. DOI: 10.5220/0002622703030308


in Bibtex Style

@conference{iceis04,
author={Miquel Montaner and Beatriz López and Josep Lluís de la Rosa},
title={EVALUATION OF RECOMMENDER SYSTEMS THROUGH SIMULATED USERS},
booktitle={Proceedings of the Sixth International Conference on Enterprise Information Systems - Volume 3: ICEIS,},
year={2004},
pages={303-308},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0002622703030308},
isbn={972-8865-00-7},
}


in EndNote Style

TY - CONF
JO - Proceedings of the Sixth International Conference on Enterprise Information Systems - Volume 3: ICEIS,
TI - EVALUATION OF RECOMMENDER SYSTEMS THROUGH SIMULATED USERS
SN - 972-8865-00-7
AU - Montaner M.
AU - López B.
AU - Lluís de la Rosa J.
PY - 2004
SP - 303
EP - 308
DO - 10.5220/0002622703030308