A Multicriteria Evaluation of Hybrid Recommender Systems: On the Usefulness of Input Data Characteristics

Reinaldo Silva Fortes, Alan R. R. de Freitas, Marcos André Gonçalves

2017

Abstract

Recommender Systems (RS) may behave differently depending on the characteristics of the input data, encouraging the development of Hybrid Filtering (HF). There are few works in the literature that explicitly characterize aspects of the input data and how they can lead to better HF solutions. Such work is limited to the scope of combination of Collaborative Filtering (CF) solutions, using only rating prediction accuracy as an evaluation criterion. However, it is known that RS also need to consider other evaluation criteria, such as novelty and diversity, and that HF involving more than one approach can lead to more effective solutions. In this work, we begin to explore this under-investigated area, by evaluating different HF strategies involving CF and Content-Based (CB) approaches, using a variety of data characteristics as extra input data, as well as different evaluation criteria. We found that the use of data characteristics in HF proved to be useful when considering different evaluation criteria. This occurs in spite of the fact that the experimented methods aim at minimizing only the rating prediction errors, without considering other criteria.

References

  1. Adar, E., Teevan, J., Dumais, S. T., and Elsas, J. L. (2009). The Web Changes Everything: Understanding the Dynamics of Web Content. In Proceedings of the Second ACM International Conference on Web Search and Data Mining, WSDM 7809, pages 282-291, New York, NY, USA. ACM.
  2. Adomavicius, G. and Zhang, J. (2012). Impact of Data Characteristics on Recommender Systems Performance. ACM Trans. Manage. Inf. Syst., 3(1):3:1-3:17.
  3. Baeza-Yates, R. A. and Ribeiro-Neto, B. (1999). Modern Information Retrieval. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA.
  4. Bao, X., Bergman, L., and Thompson, R. (2009). Stacking Recommendation Engines with Additional Metafeatures. In Proceedings of the Third ACM Conference on Recommender Systems, RecSys 7809, pages 109- 116, New York, NY, USA. ACM.
  5. Bendersky, M., Croft, W. B., and Diao, Y. (2011). Qualitybiased Ranking of Web Documents. In Proceedings of the Fourth ACM International Conference on Web Search and Data Mining, WSDM 7811, pages 95-104, New York, NY, USA. ACM.
  6. Breese, J. S., Heckerman, D., and Kadie, C. (1998). Empirical Analysis of Predictive Algorithms for Collaborative Filtering. In Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence , UAI'98, pages 43-52, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc.
  7. Burke, R. (2002). Hybrid Recommender Systems: Survey and Experiments. User Model User-Adap Inter, 12(4):331-370.
  8. Burke, R. (2007). Hybrid Web Recommender Systems. In Brusilovsky, P., Kobsa, A., and Nejdl, W., editors, The Adaptive Web, number 4321 in Lecture Notes in Computer Science, pages 377-408. Springer Berlin Heidelberg.
  9. Efron, B. and Tibshirani, R. (1986). Bootstrap Methods for Standard Errors, Confidence Intervals, and Other Measures of Statistical Accuracy. Statist. Sci., 1(1):54-75.
  10. Gantner, Z., Rendle, S., Freudenthaler, C., and SchmidtThieme, L. (2011). MyMediaLite: A Free Recommender System Library. In Proceedings of the Fifth ACM Conference on Recommender Systems, RecSys 7811, pages 305-308, New York, NY, USA. ACM.
  11. Geng, B., Li, L., Jiao, L., Gong, M., Cai, Q., and Wu, Y. (2015). NNIA-RS: A multi-objective optimization based recommender system. Physica A: Statistical Mechanics and its Applications, 424:383-397.
  12. Goldberg, K., Roeder, T., Gupta, D., and Perkins, C. (2001). Eigentaste: A constant time collaborative filtering algorithm. Inf. Retr., 4(2):133-151.
  13. Gunawardana, A. and Shani, G. (2009). A Survey of Accuracy Evaluation Metrics of Recommendation Tasks. J. Mach. Learn. Res., 10:2935-2962.
  14. Herlocker, J. L., Konstan, J. A., Borchers, A., and Riedl, J. (1999). An Algorithmic Framework for Performing Collaborative Filtering. In Proceedings of the 22Nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 7899, pages 230-237, New York, NY, USA. ACM.
  15. Herlocker, J. L., Konstan, J. A., Terveen, L. G., and Riedl, J. T. (2004). Evaluating Collaborative Filtering Recommender Systems. ACM Trans. Inf. Syst., 22(1):5- 53.
  16. Hurley, N. and Rickard, S. (2009). Comparing Measures of Sparsity. IEEE Trans. Inf. Theor., 55(10):4723-4741.
  17. Jannach, D., Zanker, M., Felfernig, A., and Friedrich, G. (2010). Recommender Systems: An Introduction. Cambridge University Press, New York.
  18. López-Ibán˜ez, M., Dubois-Lacoste, J., Pérez Cáceres, L., Birattari, M., and St ützle, T. (2016). The irace package: Iterated racing for automatic algorithm configuration. Operations Research Perspectives, 3:43-58.
  19. McCandless, M., Hatcher, E., and Gospodnetic, O. (2010). Lucene in Action, Second Edition: Covers Apache Lucene 3.0. Manning Publications Co., Greenwich, CT, USA.
  20. McNee, S. M., Riedl, J., and Konstan, J. A. (2006). Being Accurate is Not Enough: How Accuracy Metrics Have Hurt Recommender Systems. In CHI 7806 Extended Abstracts on Human Factors in Computing Systems, CHI EA 7806, pages 1097-1101, New York, NY, USA. ACM.
  21. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Louppe, G., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. (2012). Scikit-learn: Machine Learning in Python. arXiv:1201.0490 [cs]. arXiv: 1201.0490.
  22. Ribeiro, M. T., Ziviani, N., Moura, E. S. D., Hata, I., Lacerda, A., and Veloso, A. (2014). Multiobjective Pareto-Efficient Approaches for Recommender Systems. ACM Trans. Intell. Syst. Technol., 5(4):53:1- 53:20.
  23. Ricci, F., Rokach, L., and Shapira, B. (2011). Introduction to Recommender Systems Handbook. In Ricci, F., Rokach, L., Shapira, B., and Kantor, P. B., editors, Recommender Systems Handbook, pages 1-35. Springer US. DOI: 10.1007/978-0-387-85820-3 1.
  24. Sill, J., Takacs, G., Mackey, L., and Lin, D. (2009). FeatureWeighted Linear Stacking. arXiv:0911.0460 [cs]. arXiv: 0911.0460.
  25. Vargas, S. and Castells, P. (2011). Rank and Relevance in Novelty and Diversity Metrics for Recommender Systems. In Proceedings of the Fifth ACM Conference on Recommender Systems, RecSys 7811, pages 109-116, New York, NY, USA. ACM.
  26. Wang, S., Gong, M., Li, H., and Yang, J. (2016). Multiobjective optimization for long tail recommendation. Knowledge-Based Systems, 104:145-155.
  27. Zhang, M. and Hurley, N. (2008). Avoiding Monotony: Improving the Diversity of Recommendation Lists. In Proceedings of the 2008 ACM Conference on Recommender Systems, RecSys 7808, pages 123-130, New York, NY, USA. ACM.
  28. Ziegler, C.-N., McNee, S. M., Konstan, J. A., and Lausen, G. (2005). Improving Recommendation Lists Through Topic Diversification. InProceedings of the 14th International Conference on World Wide Web, WWW 7805, pages 22-32, New York, NY, USA. ACM.
  29. Zuo, Y., Gong, M., Zeng, J., Ma, L., and Jiao, L. (2015). Personalized Recommendation Based on Evolutionary Multi-Objective Optimization [Research Frontier]. IEEE Computational Intelligence Magazine, 10(1):52-62.
Download


Paper Citation


in Harvard Style

Silva Fortes R., R. R. de Freitas A. and André Gonçalves M. (2017). A Multicriteria Evaluation of Hybrid Recommender Systems: On the Usefulness of Input Data Characteristics . In Proceedings of the 19th International Conference on Enterprise Information Systems - Volume 2: ICEIS, ISBN 978-989-758-248-6, pages 623-633. DOI: 10.5220/0006315406230633


in Bibtex Style

@conference{iceis17,
author={Reinaldo Silva Fortes and Alan R. R. de Freitas and Marcos André Gonçalves},
title={A Multicriteria Evaluation of Hybrid Recommender Systems: On the Usefulness of Input Data Characteristics},
booktitle={Proceedings of the 19th International Conference on Enterprise Information Systems - Volume 2: ICEIS,},
year={2017},
pages={623-633},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006315406230633},
isbn={978-989-758-248-6},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 19th International Conference on Enterprise Information Systems - Volume 2: ICEIS,
TI - A Multicriteria Evaluation of Hybrid Recommender Systems: On the Usefulness of Input Data Characteristics
SN - 978-989-758-248-6
AU - Silva Fortes R.
AU - R. R. de Freitas A.
AU - André Gonçalves M.
PY - 2017
SP - 623
EP - 633
DO - 10.5220/0006315406230633