EVALUATION OF COLLECTING REVIEWS IN CENTRALIZED ONLINE REPUTATION SYSTEMS

Ling Liu, Malcolm Munro, William Song

2010

Abstract

Background: Centralized Online Reputation Systems (ORS) have been widely used by internet companies. They collect users’ opinions on products, transactions and events as reputation information then aggregate and publish the information to the public. Aim: Studies of reputation systems evaluation to date have tended to focus on isolated systems or their aggregating algorithms only. This paper proposes an evaluation mechanism to measure different reputation systems in the same context. Method: Reputation systems naturally have differing interfaces, and track different aspects of user behavior, however, from information system perspective, they all share five underlying components: Input, Processing, Storage, Output and Feedback Loop. Therefore, reputation systems can be divided into these five components and measured by their properties respectively. Results: The paper concentrates on the evaluation of Input and develops a set of simple formulas to represent the cost of reputation information collection. This is then applied to three different sites and the resulting analysis shows the pros and cons of the differing approaches of each of these sites.

References

  1. Chevalier, J. A. and Mayzlin, D. (2006). The effect of word of mouth on sales: Online book reviews. Journal of Marketing Research (JMR), 43(3):345-354.
  2. David, S. and Pinch, T. (2005). Six degrees of reputation: The use and abuse of online review and recommendation systems. Social Science Research Network Working Paper Series.
  3. Dellarocas, C. (2001). Analyzing the economic efficiency of ebay-like online reputation reporting mechanisms. In EC 7801: Proceedings of the 3rd ACM conference on Electronic Commerce, pages 171-179. ACM.
  4. Dellarocas, C. (2003). The digitization of word of mouth: Promise and challenges of online feedback mechanisms. Manage. Sci., 49(10):1407-1424.
  5. Falcone, R. and Castelfranchi, C. (2001). Social trust: A cognitive approach. In Castelfranchi, C. and Tan, Y.-H., editors, Trust and deception in virtual societies, pages 55-90. Kluwer Academic Publishers, Norwell, MA, USA.
  6. Friedman, E., Resnick, P., and Sami, R. (2007). Manipulation-resistant reputation systems. In Nisan, N., Roughgarden, T., Tardos, E., and Vazirani, V. V., editors, Algorithmic Game Theory, pages 677-697. Cambridge University Press.
  7. Hansen, J. P., Johansen, A. S., Hansen, D. W., and Itoh, K. (2003). Command without a click: Dwell time typing by mouse and gaze selections. In Rauterberg, M., editor, Proceedings of Human-Computer Interaction - Interaction'03, pages 121-128. IOS Press.
  8. Jøsang, A., Ismail, R., and Boyd, C. (2007). A survey of trust and reputation systems for online service provision. Decis. Support Syst., 43(2):618-644.
  9. Karat, C.-M., Halverson, C., Horn, D., and Karat, J. (1999). Patterns of entry and correction in large vocabulary continuous speech recognition systems. In CHI 7899: Proceedings of the SIGCHI conference on Human factors in computing systems, pages 568-575. ACM Press.
  10. Laat, P. (2005). Trusting virtual trust. Ethics and Information Technology, 7(3):167-180.
  11. Lampe, C. and Resnick, P. (2004). Slash(dot) and burn: distributed moderation in a large online conversation space. In CHI 7804: Proceedings of the SIGCHI conference on Human factors in computing systems, pages 543-550. ACM.
  12. Lerman, K. (2007). Social information processing in news aggregation. IEEE Internet Computing, 11(6):16-28.
  13. Liang, Z. and Shi, W. (2005). Performance evaluation of rating aggregation algorithms in reputation systems. In International Conference on Collaborative Computing: Networking, Applications and Worksharing, page 10 pp. IEEE Computer Society.
  14. Resnick, P., Zeckhauser, R., Swanson, J., and Lockwood, K. (2006). The value of reputation on ebay: A controlled experiment. Experimental Economics, 9(2):79-101.
  15. Ruohomaa, S., Kutvonen, L., and Koutrouli, E. (2007). Reputation management survey. In ARES 7807: Proceedings of the The Second International Conference on Availability, Reliability and Security, pages 103-111, Washington, DC, USA. IEEE Computer Society.
  16. Sabater, J. and Sierra, C. (2005). Review on computational trust and reputation models. Artificial Intelligence Review, 24(1):33-60.
  17. Surowiecki, J. (2005). The Wisdom of Crowds. Anchor.
  18. Weinreich, H., Obendorf, H., Herder, E., and Mayer, M. (2008). Not quite the average: An empirical study of web use. ACM Trans. Web, 2(1):1-31.
  19. Zacharia, G. and Maes, P. (2000). Trust management through reputation mechanisms. Applied Artificial Intelligence, 14(9):881-907.
Download


Paper Citation


in Harvard Style

Liu L., Munro M. and Song W. (2010). EVALUATION OF COLLECTING REVIEWS IN CENTRALIZED ONLINE REPUTATION SYSTEMS . In Proceedings of the 6th International Conference on Web Information Systems and Technology - Volume 2: WEBIST, ISBN 978-989-674-025-2, pages 281-286. DOI: 10.5220/0002763802810286


in Bibtex Style

@conference{webist10,
author={Ling Liu and Malcolm Munro and William Song},
title={EVALUATION OF COLLECTING REVIEWS IN CENTRALIZED ONLINE REPUTATION SYSTEMS},
booktitle={Proceedings of the 6th International Conference on Web Information Systems and Technology - Volume 2: WEBIST,},
year={2010},
pages={281-286},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0002763802810286},
isbn={978-989-674-025-2},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 6th International Conference on Web Information Systems and Technology - Volume 2: WEBIST,
TI - EVALUATION OF COLLECTING REVIEWS IN CENTRALIZED ONLINE REPUTATION SYSTEMS
SN - 978-989-674-025-2
AU - Liu L.
AU - Munro M.
AU - Song W.
PY - 2010
SP - 281
EP - 286
DO - 10.5220/0002763802810286