Are Non-Standard Neural Behaviors Computationally Relevant?

Stylianos Kampakis

2014

Abstract

An idea that has recently appeared in the neural network community is that networks with heterogeneous neurons and non-standard neural behaviors can provide computational advantages. A theoretical investigation of this idea was given by Kampakis (2013) for spiking neurons. In artificial neural networks this idea has been recently researched through Neural Diversity Machines (Maul, 2013). However, this idea has not been tested experimentally for spiking neural networks. This paper provides a first experimental investigation of whether neurons with non-standard behaviors can provide computational advantages. This is done by using a spiking neural network with a biologically realistic neuron model that is tested on a supervised learning task. In the first experiment the network is optimized for the supervised learning task by adjusting the parameters of the neurons in order to adapt the neural behaviors. In the second experiment, the parameter optimization is used in order to improve the network’s performance after the weights have been trained. The results confirm that neurons with non-standard behaviors can provide computational advantages for a network. Further implications of this study and suggestions for future research are discussed.

References

  1. Achard, P. & Schutter, E. D., 2006. Complex parameter landscape for a complex neuron model. PLoS Computational Biology, 2(7).
  2. Bohte, S., Kok, J. & Poutre, H. L., 2002. Error backpropagationin temporally encoded networks ofspiking neurons.. Neurocomputing, Volume 48, pp. 17-37.
  3. Bohte, S. M., Poutre, H. L. & Kok, J. N., 2001. Unsupervised clustering with spiking neurons by sparse temporal coding and multi-layer RBF networks. IEEE Transactions on Neural Networks, Volume XX.
  4. Buzsaki, G., Geisler, C., Henze, D. A. & Wang, X. J., 2004. Interneuron diversity series: circuit complexity and axon wiring economy of cortical interneurons. Trends in Neurosciences, 27(4), p. 186-193.
  5. Cohen, S. & Intrator, N., 2002. A hybrid projection-based and radial basis function architecture: initial values and global optimisation. Pattern Analysis and Applications, 5(2), pp. 113-120.
  6. Fisher, R. A., 1936. The use of multiple measurements in taxonomic problems. Annals of Eugenics, pp. 179-188.
  7. Ghosh-Dastidar, S. & Adeli, H., 2009. A new supervised learning algorithm for multiple spiking neural networks with application in epilepsy and seizure detection. Neural Networks, Volume 22.
  8. Ianella, N. & Back, A. D., 2001. A spiking neural network architecture for nonlinear function approximation. Neural Networks, 14(2001), pp. 933-939.
  9. Izhikevich, E., 2003. Simple model of spiking neurons. IEEE Transactions on Neural Networks, 14(6).
  10. Izhikevich, E. M., 2004. Which model to use for cortical spiking neurons?. IEEE Transactions on Neural Networks, 15(5), pp. 1063-1070.
  11. Izhikevich, E. M., 2006. Dynamical Systems in Neuroscience: The Geometry of Excitability and Bursting. s.l.:MIT Press.
  12. Kampakis, S., 2011. Improved Izhikevich neurons for spiking neural networks. Journal of Soft Computing.
  13. Kampakis, S., 2013 (under review). ReSpiN: A Supervised Training Algorithm for Rebound Spiking Neurons. Journal of Soft Computing.
  14. Kampakis, S., 2013. Investigating the computational power of spiking neurons with non-standard behaviors. Neural Networks, Volume 43, pp. 41-54.
  15. Keren, N., Peled, N. & Korngreen A., 2006. Constraining compartmental models using multiple voltage recordings and genetic algorithms. Journal of Neurophysiology, pp. 3730-3742.
  16. Klausberger, T. & Somogyi, P., 2008. Neuronal diversity and temporal dynamics: the unity of hippocampal circuit operations. Science, 321(5885), pp. 53-57.
  17. Maass, W., 1997. Networks of spiking neurons: the third generation of spiking neural networks. Neural Networks, 10(9), pp. 1659-1671.
  18. Maul, T., 2013 (in press, accepted manuscript). Early experiments with neural diversity machines.
  19. Meftah, B., Lezoray, O. & Benyettou, A., 2010. Segmentation and edge detection based on spiking neural network model. Neural Processing Letters, 32(2), pp. 131-146.
  20. Moore, C. I., Carlen, M., Knoblich, U. & Cardin, J. A., 2010. Neocortical interneurons: from diversity, strength. Cell, 142(2), pp. 189-193.
  21. Potjans, W., Morrison, A. & Diesmann, M., 2009. A spiking neural network model of an actor-critic learning agent. Neural Computation, 21(2), pp. 301- 339.
  22. Rawlins, G. J. E. ed., 1991. Foundations of Genetic Algorithms (FOGA 1). s.l.:Morgan Kaufmann.
  23. Taylor, A. M. & Enoka, R. M., 2004. Optimization of input patterns and neuronal properties to evoke motor neuron synchronization. Journal of Computational Neuroscience, 16(2), pp. 139-157.
  24. Tutkun, N., 2009. Parameter estimation in mathematical models using the real coded genetic algorithms. Expert Systems with Applications, 36(2), pp. 3342-3345.
  25. Valko, M., Marques, N. C. & Castellani, M., 2005. Evolutionary feature selection for spiking neural network pattern classifiers. Covilha, IEEE, pp. 181 - 187.
  26. Van Geit, W., De Schutter, E. & Achard, P., 2008. Automated neuron model optimization techniques: a review. Biological Cybernetics, Volume 99, pp. 241- 251.
  27. Wade, J. J., McDaid, L. J., Santos, J. A. & Sayers, H. M., 2008. SWAT: An unsupervised SNN training algorithm for classification problems. Hong Kong, IEEE, pp. 2648 - 2655.
  28. Wang, H., 2009. Improvement of Izhikevich's neuronal and neural network model. Wuhan, China, IEEE.
  29. Wu, C.-H., Tzeng, G.-H. & Lin, R.-H., 2009. A novel hybrid genetic algorithm for kernel function and parameter optimization in support vector regression. Expert Systems with Applications, 36(3), pp. 4725- 4735.
Download


Paper Citation


in Harvard Style

Kampakis S. (2014). Are Non-Standard Neural Behaviors Computationally Relevant? . In Proceedings of the International Conference on Neural Computation Theory and Applications - Volume 1: NCTA, (IJCCI 2014) ISBN 978-989-758-054-3, pages 32-37. DOI: 10.5220/0005030400320037


in Bibtex Style

@conference{ncta14,
author={Stylianos Kampakis},
title={Are Non-Standard Neural Behaviors Computationally Relevant?},
booktitle={Proceedings of the International Conference on Neural Computation Theory and Applications - Volume 1: NCTA, (IJCCI 2014)},
year={2014},
pages={32-37},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005030400320037},
isbn={978-989-758-054-3},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Neural Computation Theory and Applications - Volume 1: NCTA, (IJCCI 2014)
TI - Are Non-Standard Neural Behaviors Computationally Relevant?
SN - 978-989-758-054-3
AU - Kampakis S.
PY - 2014
SP - 32
EP - 37
DO - 10.5220/0005030400320037