Does Ventriloquism Aftereffect Transfer across Sound Frequencies? - A Theoretical Analysis via a Neural Network Model

Elisa Magosso, Filippo Cona, Cristiano Cuppini, Mauro Ursino

2013

Abstract

When an auditory stimulus and a visual stimulus are simultaneously presented in spatial disparity, the sound is perceived shifted toward the visual stimulus (ventriloquism effect). After adaptation to a ventriloquism situation, enduring sound shifts are observed in the absence of the visual stimulus (ventriloquism aftereffect). Experimental studies report discordant results as to aftereffect generalization across sound frequencies, varying from aftereffect staying confined to the sound frequency used during the adaptation, to aftereffect transferring across some octaves of frequency. Here, we present a model of visual-auditory interactions, able to simulate the ventriloquism effect and to reproduce – via Hebbian plasticity rules – the ventriloquism aftereffect. The model is suitable to investigate aftereffect generalization as the simulated auditory neurons code both for spatial and spectral properties of the auditory stimuli. The model provides a plausible hypothesis to interpret the discordant results in the literature, showing that different sound intensities may produce different extents of aftereffect generalization. Model mechanisms and hypotheses are discussed in relation to the neurophysiological and psychophysical literature.

References

  1. Bertelson, P. and Radeau, M. (1981). Cross-modal bias and perceptual fusion with auditory-visual spatial discordance. Perception & Psychophysics, 29(6), pp.578-584.
  2. Bertelson, P. A., G. (1998). Automatic visual bias of perceived auditory location. Psychonomic Bullettin & Review, 5(3), pp.482-489.
  3. Cuppini, C., Magosso, E., Rowland, B., Stein, B. and Ursino, M. (2012). Hebbian mechanisms help explain development of multisensory integration in the superior colliculus: a neural network model. Biological Cybernetics, 106(11-12), pp.691-713.
  4. Cuppini, C., Stein, B. E., Rowland, B. A., Magosso, E. and Ursino, M. (2011). A computational study of multisensory maturation in the superior colliculus (SC). Experimental Brain Research, 213(2-3), pp.341- 349.
  5. Ernst, M. O. and Bulthoff, H. H. (2004). Merging the senses into a robust percept. Trends in Cognitive Sciences, 8(4), pp.162-169.
  6. Frissen, I., Vroomen, J., De Gelder, B. and Bertelson, P. (2003). The aftereffects of ventriloquism: are they sound-frequency specific? Acta Psychologica, 113(3), pp.315-327.
  7. Frissen, I., Vroomen, J., De Gelder, B. and Bertelson, P. (2005). The aftereffects of ventriloquism: generalization across sound-frequencies. Acta Psychologica, 118(1-2), pp.93-100.
  8. Fukushima, K. (1980). Neocognitron: a self organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological Cybernetics, 36(4), pp.193-202.
  9. Haykin, S. (1994). Neural Networks: A Comprehensive Foundation, New York, Macmillan College Publishing Company.
  10. Kohonen, T. (1982). Self-Organized Formation of Topologically Correct Feature Maps. Biological Cybernetics, 43, pp.59-69.
  11. Kohonen, T. (1995). Sel-Organizing Maps, Berlin, Springer-Verlag.
  12. Kohonen, T. and Hari, R. (1999). Where the abstract feature maps of the brain might come from. Trends in Neurosciences, 22(3), pp.135-139.
  13. Lewald, J. (2002). Rapid adaptation to auditory-visual spatial disparity. Learning & Memory, 9(5), pp.268- 278.
  14. Magosso, E. (2010). Integrating information from vision and touch: a neural network modeling study. IEEE Transactions on Information Technology in Biomedicine, 14(3), pp.598-612.
  15. Magosso, E., Cuppini, C., Serino, A., Di Pellegrino, G. and Ursino, M. (2008). A theoretical study of multisensory integration in the superior colliculus by a neural network model. Neural Networks, 21(6), pp.817-829.
  16. Magosso, E., Cuppini, C. and Ursino, M. (2012). A neural network model of ventriloquism effect and aftereffect. PloS one, 7(8), e42503, pp.1-19.
  17. Magosso, E., Serino, A., Di Pellegrino, G. and Ursino, M. (2010a). Crossmodal links between vision and touch in spatial attention: a computational modelling study. Computational Intelligence and Neuroscience, ID 304941, pp.1-13.
  18. Magosso, E., Ursino, M., Di Pellegrino, G., Ladavas, E. and Serino, A. (2010b). Neural bases of peri-hand space plasticity through tool-use: insights from a combined computational-experimental approach. Neuro-psychologia, 48(3), pp.812-830.
  19. Magosso, E., Zavaglia, M., Serino, A., Di Pellegrino, G. and Ursino, M. (2010c). Visuotactile representation of peripersonal space: a neural network study. Neural Computation, 22(1), pp.190-243.
  20. Miyake, S. and Fukushima, K. (1984). A neural network model for the mechanism of feature-extraction. A selforganizing network with feedback inhibition. Biological Cybernetics, 50(5), pp.377-384.
  21. Rajan, R., Aitkin, L. M. and Irvine, D. R. (1990). Azimuthal sensitivity of neurons in primary auditory cortex of cats. II. Organization along frequency-band strips. Journal of Neurophysiology, 64(3), pp.888-902.
  22. Recanzone, G. H. (1998). Rapidly induced auditory plasticity: the ventriloquism aftereffect. Proceedings of the National Academy of Sciences of the United States of America, 95(3), pp.869-875.
  23. Recanzone, G. H. (2000). Spatial processing in the auditory cortex of the macaque monkey. Proceedings of the National Academy of Sciences of the United States of America, 97(22), pp.11829-11835.
  24. Recanzone, G. H. (2009). Interactions of auditory and visual stimuli in space and time. Hearing Research, 258(1-2), pp.89-99.
  25. Recanzone, G. H., Guard, D. C. and Phan, M. L. (2000). Frequency and intensity response properties of single neurons in the auditory cortex of the behaving macaque monkey. Journal of Neurophysiology, 83(4), pp.2315-2331.
  26. Ritter, H. (1990). Self-organizing maps for internal representations. Psychological Research, 52(2-3), pp.128-136.
  27. Slutsky, D. A. and Recanzone, G. H. (2001). Temporal and spatial dependency of the ventriloquism effect. Neuroreport, 12(1), pp.7-10.
  28. Ursino, M., Cuppini, C., Magosso, E., Serino, A. and Di Pellegrino, G. (2009). Multisensory integration in the superior colliculus: a neural network model. Journal of Computational Neuroscience, 26(1), pp.55-73.
  29. Welch, R. B. and Warren, D. H. (1980). Immediate perceptual response to intersensory discrepancy. Psychological Bulletin, 88(3), pp.638-667.
  30. Woods, T. M., Lopez, S. E., Long, J. H., Rahman, J. E. and Recanzone, G. H. (2006). Effects of stimulus azimuth and intensity on the single-neuron activity in the auditory cortex of the alert macaque monkey. Journal of Neurophysiology, 96(6), pp.3323-3337.
  31. Woods, T. M. and Recanzone, G. H. (2004). Visually induced plasticity of auditory spatial perception in macaques. Current Biology, 14(17), pp.1559-1564.
  32. Xerri, C. (2012). Plasticity of cortical maps: multiple triggers for adaptive reorganization following brain damage and spinal cord injury. The Neuroscientist, 18(2), pp.133-148.
Download


Paper Citation


in Harvard Style

Magosso E., Cona F., Cuppini C. and Ursino M. (2013). Does Ventriloquism Aftereffect Transfer across Sound Frequencies? - A Theoretical Analysis via a Neural Network Model . In Proceedings of the 5th International Joint Conference on Computational Intelligence - Volume 1: NCTA, (IJCCI 2013) ISBN 978-989-8565-77-8, pages 360-369. DOI: 10.5220/0004551403600369


in Bibtex Style

@conference{ncta13,
author={Elisa Magosso and Filippo Cona and Cristiano Cuppini and Mauro Ursino},
title={Does Ventriloquism Aftereffect Transfer across Sound Frequencies? - A Theoretical Analysis via a Neural Network Model},
booktitle={Proceedings of the 5th International Joint Conference on Computational Intelligence - Volume 1: NCTA, (IJCCI 2013)},
year={2013},
pages={360-369},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004551403600369},
isbn={978-989-8565-77-8},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 5th International Joint Conference on Computational Intelligence - Volume 1: NCTA, (IJCCI 2013)
TI - Does Ventriloquism Aftereffect Transfer across Sound Frequencies? - A Theoretical Analysis via a Neural Network Model
SN - 978-989-8565-77-8
AU - Magosso E.
AU - Cona F.
AU - Cuppini C.
AU - Ursino M.
PY - 2013
SP - 360
EP - 369
DO - 10.5220/0004551403600369