Natural Language Interface Put in Perspective: Interaction of Search Method and Task Complexity

QianYing Wang, Jiang Hu, Clifford Nass

2005

Abstract

A 2x2 mixed design experiment (N=52) was conducted to examine the effects of search method and task complexity on users’ information-seeking performance and affective experience in an e-commerce context. The former factor had two within-participants conditions: keyword (KW) vs. natural language (NL) search; the latter factor had two between-participants conditions: simple vs. complex tasks. The results show that participants in the complex task condition were more successful when they used KW search than NL search. They thought the tasks were less difficult and reported more enjoyment and confidence with KW search. In the meantime, simple task participants performed better when they used NL rather than KW search. They also perceived the tasks as easier and more enjoyable, and had higher levels of confidence with the results, when NL was used. The findings suggest that NL search is not the panacea for all information retrieval tasks, depending on the complexity of task. Implications for interface design and directions for future research are discussed.

References

  1. Laurel, B.: Interface Agents: Metaphors with Character. In Laurel, B. (ed.), The Art of Human-Computer Interface Design. Addison-Wesley, Reading, MA (1990)
  2. Blackmon, M.H., Polson, G.P., Kitajima, M, Lewis C.: Design Methods: Cognitive Walkthrough for the Web. Proceedings of CHI 2002. ACM Press, 463 - 470
  3. Borgman, C.L., Belkin, N.J., Croft, W.B., Lesk, M.E., Landauer, T.K.: Retrieval Systems for the Information Seeker: Can the Role of Intermediary be Automated? Proceedings of CHI 7888 (Washington, D.C.), ACM Press, 51-53
  4. Card, S., Pirolli, P., Van Der Wege, M., Morrison, J., Reeder, R., Schraedley, P., Boshart, J.: Information Scent as a Driver of Web Behavior Graphs: Results of a Protocol Analysis Method for Web Usability. Proceedings of CHI 2001. ACM Press, 498-505
  5. Chi, E., Pirolli, P., Chen, K., Pitkow, J. E.: Using Information Scent to Model User Needs and Actions on the Web. Proceedings of CHI 2001. ACM Press, 490-497
  6. Larson, K., Czerwinski, M.: Web Page Design: Implications of Memory, Structure and Scent for Information Retrieval. Proceedings of CHI 7898. ACM Press, 25-32
  7. Nielsen, J.: Designing Web Usability. New Riders, Indianapolis, IN (1999)
  8. Pirolli, P.: Computational Models of Information Scent-Following in a Very Large Browsable Text Collection. Proceedings of CHI 7897. ACM Press, 3-10
  9. Pirolli, P., Card, S. K.: Information Foraging. Psychological Review 106(4) (1999), 643- 675
  10. Sellen A.J., Murphy R., Shaw K.L.: How Knowledge Workers Use the Web. Proceedings of CHI 2002. ACM Press, 227-234
  11. Spool, J.M., Scanlon, T., Schroeder, W., Snyder, C., DeAngelo, T.: Web Site Usability. Morgan Kaufman, San Francisco, CA (1999)
  12. Doszkocs T.: Natural Language Processing in Intelligent Information Retrieval. Proceedings of the 1985 ACM Annual Conference on The Range of Computing: Mid-80's Perspective. ACM Press, 356-359
  13. Guglielmo E.J., Rowe N.C.: Natural-Language Retrieval of Images Based on Descriptive Captions. ACM Transactions on Information Systems (TOIS), July 1996, Vol. 14, Issue 3, 237-267
  14. Jacob, P. S., Rau, L.F.: Natural Language Techniques for Intelligent Information Retrieval. Proceedings of the Eleventh International Conference on Research & development in Information Retrieval (1988), 85-99
  15. Meng, F.: A Natural Language Interface for Information Retrieval from Forms on the World Wide Web. Proceeding of the 20th International Conference on Information Systems (1999), 540-545
  16. Vivacqua A., Lieberman H.: Agents to Assist in Finding Help. Proceedings of CHI 2000. ACM Press, 65-72
  17. Griffith, C.: WESTLAW's WIN: Not Only Natural, But New. Information Today, 9-11, October 1992.
  18. Ask Jeeves, Inc. http://www.ask.com
  19. Turtle H.: Natural Language vs. Boolean Query Evaluation: A Comparison of Retrieval Performance. Proceedings of the Seventeenth Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval (1994), 212-220
  20. Conversagent, Inc. http://www.conversagent.com
  21. Clark, H.H.: Using Language. Cambridge University Press, New York (1996)
  22. Brennan, S.: Conversation with and through Computers. User Modeling and User-Adapted Interaction, 1 (1991), 67-86
  23. Pearson, J., Pickering, M.J. Branigan, H.P., McLean, J.F., Nass, C.I., Hu, J.: The Influence of Beliefs about an Interlocutor on Lexical and Syntactic Alignment: Evidence from Human-Computer Dialogues. Proceeding of the 10th Annual Architectures and Mechanisms of Language Processing Conference (2004)
  24. Nass, C.I., Hu, J., Pearson, J., Pickering, M.J., Branigan, H.P.: Linguistic Alignment in HCI vs. CMC: Do Users Mimic Conversation Partners' Grammar and Word Choices? Unpublished manuscript (2004).
  25. Zoltan-Ford, E.: How to Get People to Say and Type What Computers Can Understand. International Journal of Man-Machine Studies, 34(4) (1991), 527-547
Download


Paper Citation


in Harvard Style

Wang Q., Hu J. and Nass C. (2005). Natural Language Interface Put in Perspective: Interaction of Search Method and Task Complexity . In Proceedings of the 2nd International Workshop on Natural Language Understanding and Cognitive Science - Volume 1: NLUCS, (ICEIS 2005) ISBN 972-8865-23-6X, pages 3-12. DOI: 10.5220/0002566000030012


in Bibtex Style

@conference{nlucs05,
author={QianYing Wang and Jiang Hu and Clifford Nass},
title={Natural Language Interface Put in Perspective: Interaction of Search Method and Task Complexity},
booktitle={Proceedings of the 2nd International Workshop on Natural Language Understanding and Cognitive Science - Volume 1: NLUCS, (ICEIS 2005)},
year={2005},
pages={3-12},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0002566000030012},
isbn={972-8865-23-6X},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 2nd International Workshop on Natural Language Understanding and Cognitive Science - Volume 1: NLUCS, (ICEIS 2005)
TI - Natural Language Interface Put in Perspective: Interaction of Search Method and Task Complexity
SN - 972-8865-23-6X
AU - Wang Q.
AU - Hu J.
AU - Nass C.
PY - 2005
SP - 3
EP - 12
DO - 10.5220/0002566000030012