Towards a System Architecture for Integrating Cross-modal Context in Syntactic Disambiguation

Patrick McCrae, Wolfgang Menzel

Abstract

Most natural language utterances are inherently ambiguous, which results in semantic under specification. Yet, despite the omnipresence of ambiguity, human communication still succeeds in most cases and even displays a remarkable robustness – quite in contrast to the majority of natural language applications today. The reason for this is that in processing an ambiguous utterance humans also integrate information from sources other than the utterance itself and thus have access to additional knowledge to enrich the semantic specification that guides disambiguation. One important such source of additional semantic knowledge in humans is sensory input from cross-modal perception. In this paper we describe a system architecture motivated by effects during human sentence processing which permits to study the integration of cross-modal context knowledge in syntactic parsing. We hypothesize that integrating cross-modal context into syntactic constraint dependency parsing will significantly and substantially improve the accuracy of structural ambiguity resolution.

References

  1. Knoeferle, Pia S. (2005). The Role of Visual Scenes inSpoken Language Comprehension: Evidence from Eye-Tracking (PhD Thesis). Saarbrücken: Universität desSaarlandes.
  2. Crain, Stephen & Steedman, Mark (1985). On not Being Led up the Garden Path - the Use of Context by the Psychological Syntax Parser. In D. Dowty, L. Karttunen and A.Zwicky (Eds.), Natural language parsing: Psychological, computational, and theoretical perspectives (pp. 320 - 358). Cambridge University Press.
  3. Lieberman, Henry, Faaborg, Alexander, Daher, Waseem, & Espinosa, José (2005). How to Wreck a Nice Beach You Sing Calm Incense. International Conference on Intelligent UserInterfaces, IUI 2005, January 9 - 12, 2005.
  4. Watanabe, Katsumi (2001). Cross-modal Interactions in Humans (PhD Thesis). Pasadena, California: California Instituteof Technology.
  5. Tanenhaus, Michael, Spivey-Knowlton, Michael. J., Eberhard, Kathleen M. et al. (1995). Integration of visual andlinguistic information in spoken language comprehension. SCIENCE (Volume 268), 16 June 1995, pp. 1632 - 1634.
  6. Foth, Kilian & Menzel, Wolfgang (2006). The Benefit of Stochastic PP Attachment to a Rule-Based Parser. In Proceedings of the 21st International Conference on Computational Linguistics. Sydney: Coling-ACL-2006.
  7. Lüdtke, Dirk & Sato, Satoshi (2003). Fast Base NP Chunking with Decision Trees - Experiments on Different POS Tag Settings. In A. Gelbukh (Ed.) Computational linguistics and intelligent text processing, Springer LNCS, 2003, pp. 136 - 147.
  8. Kudo, Taku & Matsumoto, Yuji (2000). Use of SupportVector Learning for Chunk Identification. In Proceedings of CoNLL-2000 and LLL-2000. Lisbon, Portugal.
  9. Auble, Pamela & Franks, Jeffery J. (1983). Sentence comprehension processes. Journal of Verbal Learning and Verbal Behavior (22), pp. 395 - 405.
  10. Grimshaw, Jane B. (1990). Argument structure. Cambridge MA: MIT Press.
  11. Simpson, Jane (1991). Warlpiri morpho-syntax: A lexicalist approach. Dordrecht: Kluwer Academic Publishers.
  12. Trueswell, John C., Tanenhaus, Michael K., & Garnsey,Susan M. (1994). Semantic Influences on Parsing: Use of Thematic Role Information in Syntactic Ambiguity Resolution. Journal of Memory and Language (33), pp. 285 - 318.
  13. Kuperberg, Gina R., Kreher, Donna A., Sitnikova, Tatiana, Caplan, David N., & Holcomb, Phillip J. (2006). The Role of Animacy and Thematic Relationships in Processing Active English Sentences: Evidence from Event-Related Potentials. In Brain and Language, in press.
  14. Dowty, David (1989). On the Semantic Content of theNotion of 'Thematic Role'. In Gennaro Chierchia, Barbara H.Partee and Raymond Turner (Eds.), Properties, types and meaning (Volume II: Semantic issues). Dordrecht: KluwerAcademic Publishers, pp. 69 - 130.
  15. Ferretti, Todd R., McRae, Ken & Hatherell, Andrea (2001). Integrating Verbs, Situation Schemas, and Thematic Role Concepts. Journal of Memory and Language (33), pp. 516 - 547.
  16. McCawley, J. D. (1968). The Role of Semantics in Grammar. In E. Bach & R. T. Harms (Eds.), Universals in linguistic theory. New York: Holt, Rinehart, & Winston.
Download


Paper Citation


in Harvard Style

McCrae P. and Menzel W. (2007). Towards a System Architecture for Integrating Cross-modal Context in Syntactic Disambiguation . In Proceedings of the 4th International Workshop on Natural Language Processing and Cognitive Science - Volume 1: NLPCS, (ICEIS 2007) ISBN 978-972-8865-97-9, pages 228-237. DOI: 10.5220/0002430602280237


in Bibtex Style

@conference{nlpcs07,
author={Patrick McCrae and Wolfgang Menzel},
title={Towards a System Architecture for Integrating Cross-modal Context in Syntactic Disambiguation},
booktitle={Proceedings of the 4th International Workshop on Natural Language Processing and Cognitive Science - Volume 1: NLPCS, (ICEIS 2007)},
year={2007},
pages={228-237},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0002430602280237},
isbn={978-972-8865-97-9},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 4th International Workshop on Natural Language Processing and Cognitive Science - Volume 1: NLPCS, (ICEIS 2007)
TI - Towards a System Architecture for Integrating Cross-modal Context in Syntactic Disambiguation
SN - 978-972-8865-97-9
AU - McCrae P.
AU - Menzel W.
PY - 2007
SP - 228
EP - 237
DO - 10.5220/0002430602280237