Shifts of Attention During Spatial Language Comprehension - A Computational Investigation

Thomas Kluth, Michele Burigo, Pia Knoeferle

2016

Abstract

Regier and Carlson (2001) have investigated the processing of spatial prepositions and developed a cognitive model that formalizes how spatial prepositions are evaluated against depicted spatial relations between objects. In their Attentional Vector Sum (AVS) model, a population of vectors is weighted with visual attention, rooted at the reference object and pointing to the located object. The deviation of the vector sum from a reference direction is then used to evaluate the goodness-of-fit of the spatial preposition. Crucially, the AVS model assumes a shift of attention from the reference object to the located object. The direction of this shift has been challenged by recent psycholinguistic and neuroscientific findings. We propose a modified version of the AVS model (the rAVS model) that integrates these findings. In the rAVS model, attention shifts from the located object to the reference object in contrast to the attentional shift from the reference object to the located object implemented in the AVS model. Our model simulations show that the rAVS model accounts for both the data that inspired the AVS model and the most recent findings.

References

  1. Burigo, M. (2008). On the role of informativeness in spatial language comprehension. PhD thesis, School of Psychology, University of Plymouth.
  2. Burigo, M. and Knoeferle, P. (2015). Visual attention during spatial language comprehension. PloS ONE, 10(1):e0115758.
  3. Burigo, M. and Sacchi, S. (2013). Object orientation affects spatial language comprehension. Cognitive Science, 37(8):1471-1492.
  4. Canty, A. and Ripley, B. (2015). boot: Bootstrap R (S-Plus) Functions. R package version 1.3-15.
  5. Carlson, L. A. and Logan, G. D. (2005). Attention and spatial language. In Itti, L., Rees, G., and Tsotsos, J. K., editors, Neurobiology of Attention, chapter 54, pages 330-336. Elsevier.
  6. Carlson, L. A., Regier, T., Lopez, W., and Corrigan, B. (2006). Attention unites form and function in spatial language. Spatial Cognition and Computation, 6(4):295-308.
  7. Carlson-Radvansky, L. A., Covey, E. S., and Lattanzi, K. M. (1999). What effects on where: Functional influences on spatial relations. Psychological Science, 10(6):516-521.
  8. Carrasco, M. (2011). Visual attention: The past 25 years. Vision Research, 51(13):1484-1525.
  9. Coventry, K. R., Lynott, D., Cangelosi, A., Monrouxe, L., Joyce, D., and Richardson, D. C. (2010). Spatial language, visual attention, and perceptual simulation. Brain and Language, 112(3):202-213.
  10. Coventry, K. R., Prat Sala, M., and Richards, L. (2001). The interplay between geometry and function in the comprehension of over, under, above, and below. Journal of Memory and Language, 44(3):376-398.
  11. Franconeri, S. L., Scimeca, J. M., Roth, J. C., Helseth, S. A., and Kahn, L. E. (2012). Flexible visual processing of spatial relationships. Cognition, 122(2):210-227.
  12. Georgopoulos, A. P., Schwartz, A. B., and Kettner, R. E. (1986). Neuronal Population Coding of Movement Direction. Science, 233:1416-1419.
  13. Hörberg, T. (2008). Influences of form and function on the acceptability of projective prepositions in swedish. Spatial Cognition & Computation, 8(3):193-218.
  14. Kim, W., Navarro, D. J., Pitt, M. A., and Myung, I. J. (2004). An MCMC-based method of comparing connectionist models in cognitive science. Advances in Neural Information Processing Systems, 16:937-944.
  15. Kluth, T. (2016). A C++ Implementation of the reversed Attentional Vector Sum (rAVS) model.
  16. Kluth, T. and Schultheis, H. (2014). Attentional distribution and spatial language. In Freksa, C., Nebel, B., Hegarty, M., and Barkowsky, T., editors, Spatial Cognition IX, volume 8684 of Lecture Notes in Computer Science, pages 76-91. Springer International Publishing.
  17. Lee, C., Rohrer, W. H., and Sparks, D. L. (1988). Population coding of saccadic eye movements by neurons in the superior colliculus. Nature, 332:357-360.
  18. Logan, G. D. (1994). Spatial attention and the apprehension of spatial relations. Journal of Experimental Psychology: Human Perception and Performance, 20(5):1015.
  19. Logan, G. D. (1995). Linguistic and conceptual control of visual spatial attention. Cognitive Psychology, 28(2):103-174.
  20. Logan, G. D. and Sadler, D. D. (1996). A computational analysis of the apprehension of spatial relations. In Bloom, P., Peterson, M. A., Nadel, L., and Garrett, M. F., editors, Language and Space, chapter 13, pages 493-530. The MIT Press.
  21. Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H., and Teller, E. (1953). Equation of state calculations by fast computing machines. The Journal of Chemical Physics, 21(6):1087-1092.
  22. Navarro, D. J., Pitt, M. A., and Myung, I. J. (2004). Assessing the distinguishability of models and the informativeness of data. Cognitive Psychology, 49(1):47-84.
  23. Pitt, M. A., Kim, W., Navarro, D. J., and Myung, J. I. (2006). Global model analysis by parameter space partitioning. Psychological Review, 113(1):57.
  24. Pitt, M. A. and Myung, I. J. (2002). When a good fit can be bad. Trends in cognitive sciences, 6(10):421-425.
  25. Regier, T. and Carlson, L. A. (2001). Grounding spatial language in perception: An empirical and computational investigation. Journal of Experimental Psychology: General, 130(2):273-298.
  26. Roberts, S. and Pashler, H. (2000). How persuasive is a good fit? A comment on theory testing. Psychological review, 107(2):358-367.
  27. Roth, J. C. and Franconeri, S. L. (2012). Asymmetric coding of categorical spatial relations in both language and vision. Frontiers in Psychology, 3(464).
  28. Schultheis, H., Singhaniya, A., and Chaplot, D. S. (2013). Comparing model comparison methods. In Proceedings of the 35th Annual Conference of the Cognitive Science Society, pages 1294 - 1299, Austin, TX. Cognitive Science Society.
  29. Wagenmakers, E.-J., Ratcliff, R., Gomez, P., and Iverson, G. J. (2004). Assessing model mimicry using the parametric bootstrap. Journal of Mathematical Psychology, 48(1):28-50.
Download


Paper Citation


in Harvard Style

Kluth T., Burigo M. and Knoeferle P. (2016). Shifts of Attention During Spatial Language Comprehension - A Computational Investigation . In Proceedings of the 8th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, ISBN 978-989-758-172-4, pages 213-222. DOI: 10.5220/0005851202130222


in Bibtex Style

@conference{icaart16,
author={Thomas Kluth and Michele Burigo and Pia Knoeferle},
title={Shifts of Attention During Spatial Language Comprehension - A Computational Investigation},
booktitle={Proceedings of the 8th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},
year={2016},
pages={213-222},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005851202130222},
isbn={978-989-758-172-4},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 8th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,
TI - Shifts of Attention During Spatial Language Comprehension - A Computational Investigation
SN - 978-989-758-172-4
AU - Kluth T.
AU - Burigo M.
AU - Knoeferle P.
PY - 2016
SP - 213
EP - 222
DO - 10.5220/0005851202130222