Parallel Implementation of Spatial Pooler in Hierarchical Temporal Memory

Marcin Pietron, Maciej Wielgosz, Kazimierz Wiatr

2016

Abstract

Hierarchical Temporal Memory is a structure that models some of the structural and algorithmic properties of the neocortex. HTM is a biological model based on the memory-prediction theory of brain. HTM is a method for discovering and learning of observed input patterns and sequences, building an increasingly complex models. HTM combines and extends approaches used in sparse distributed memory, bayesian networks, spatial and temporal clustering algorithms, using a tree-shaped hierarchy neural networks. It is quite a new model of deep learning process, which is very efficient technique in artificial intelligence algorithms. HTM like other deep learning models (Boltzmann machine, deep belief networks etc.) has structure which can be efficiently processed by parallel machines. Modern multi-core processors with wide vector processing units (SSE, AVX), GPGPU are platforms that can tremendously speed up learning, classifying or clustering algorithms based on deep learning models (e.g. Cuda Toolkit 7.0). The current bottleneck of this new flexible artifficial intelligence model is efficiency. This article focuses on parallel processing of HTM learning algorithms in parallel hardware platforms. This work is the first one about implementation of HTM architecture and its algorithms in hardware accelerators. The article doesn’t study quality of the algorithm.

References

  1. Cai, X., Xu, Z., Lai, G., Wu, C., and Lin, X. (2012). Gpuaccelerated restricted boltzmann machine for collaborative filtering. ICA3PP'12 Proceedings of the 12th international conference on Algorithms and Architectures for Parallel Processing - Volume Part I, pages 303-316.
  2. Cisco (2012). Visual networking index. Visual Networking Index, Cisco Systems.
  3. Hawkins, J. and Ahmad, S. (2011). Numenta, white paper. Numenta, Hierachical Temporal Memory, white paper, version 0.2.1, september 12, 2011.
  4. Hilbert, M. and Lopez, P. (2011). The worlds technological capacity to store, communicate, and compute information. In University of Vermont, Vol. 332, no. 6025, pages 60-65.
  5. Idc (2011). Idc predicts. IDC Predicts 2012 Will Be the Year of Mobile and Cloud Platform Wars as IT Vendors Vie for Leadership While the Industry Redefines Itself. IDC. 2011-12-01.
  6. Kapuscinski, T. (2010). Using hierarchical temporal memory for vision-based hand shape recognition under large variations in hands rotation. 10th International Conference, ICAISC 2010, Zakopane, Poland, in Artifical Intelligence and Soft Computing, SpringerVerlag, pages 272-279.
  7. Lopes, N., Ribeiro, B., and Goncalves, J. (2012). Restricted boltzmann machines and deep belief networks on multi-core processors. Neural Networks (IJCNN), The 2012 International Joint Conference on, pages 1- 7.
  8. Numenta (2011). Nupic https://github.com/numenta/nupic/wiki.
  9. NVIDIA (2014). Cuda https://developer.nvidia.com/cuda-gpus.
  10. OpenMP (2010). Openmp library. http://www.openmp.org.
  11. Sherwin, J. and Mavris, D. (2009). Hierarchical temporal memory algorithms for understanding asymmetric warfare. Aerospace conference, 2009 IEEE, MT, pages 1-10.
  12. Vyas, P. and Zaveri, M. (2013). Verilog implementation of a node of hierarchical temporal memory. Asian Journal of Computer Science and Information Technology, AJCSIT, 3:103-108.
  13. Wu, X., Zhu, X., Wu, G.-Q., and Ding, W. (2014). Data mining with big data. Knowledge and Data Engineering, IEEE Transactions on, 26(1):97-107.
Download


Paper Citation


in Harvard Style

Pietron M., Wielgosz M. and Wiatr K. (2016). Parallel Implementation of Spatial Pooler in Hierarchical Temporal Memory . In Proceedings of the 8th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, ISBN 978-989-758-172-4, pages 346-353. DOI: 10.5220/0005706603460353


in Bibtex Style

@conference{icaart16,
author={Marcin Pietron and Maciej Wielgosz and Kazimierz Wiatr},
title={Parallel Implementation of Spatial Pooler in Hierarchical Temporal Memory},
booktitle={Proceedings of the 8th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},
year={2016},
pages={346-353},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005706603460353},
isbn={978-989-758-172-4},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 8th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,
TI - Parallel Implementation of Spatial Pooler in Hierarchical Temporal Memory
SN - 978-989-758-172-4
AU - Pietron M.
AU - Wielgosz M.
AU - Wiatr K.
PY - 2016
SP - 346
EP - 353
DO - 10.5220/0005706603460353