The Integer Approximation of Undirected Graphical Models

Nico Piatkowski, Sangkyun Lee, Katharina Morik

2014

Abstract

Machine learning on resource constrained ubiquitous devices suffers from high energy consumption and slow execution time. In this paper, it is investigated how to modify machine learning algorithms in order to reduce the number of consumed clock cycles—not by reducing the asymptotic complexity, but by assuming a weaker execution platform. In particular, an integer approximation to the class of undirected graphical models is proposed. Algorithms for inference, maximum-a-posteriori prediction and parameter estimation are presented and approximation error is discussed. In numerical evaluations on synthetic data, the response of the model to several influential properties of the data is investigated. The results on the synthetic data are confirmed with a natural language processing task on an open data set. In addition, the runtime on low-end hardware is regarded. The overall speedup of the new algorithms is at least 2× while overall loss in accuracy is rather small. This allows running probabilistic methods on very small devices, even if they do not contain a processor that is capable of executing floating point arithmetic at all.

References

  1. Ahmadi, B., Kersting, K., and Natarajan, S. (2012). Lifted online training of relational models with stochastic gradient methods. In Machine Learning and Knowledge Discovery in Databases, volume 7523 of Lecture Notes in Computer Science, pages 585-600.
  2. Blitzstein, J. and Diaconis, P. (2011). A sequential importance sampling algorithm for generating random graphs with prescribed degrees. Internet Mathmatics, 6(4):489-522.
  3. Choirat, C. and Seri, R. (2012). Estimation in discrete parameter models. Statistical Science, 27(2):278-293.
  4. Hassibi, A. and Boyd, S. (1998). Integer parameter estimation in linear models with applications to gps. IEEE Transactions on Signal Processing, 46(11):2938- 2952.
  5. Ihler, A. and McAllester, D. (2009). Particle belief propagation. In Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics, pages 256-263, JMLR: W&CP 5.
  6. Ihler, A. T., III, J. W. F., and Willsky, A. S. (2005). Loopy belief propagation: Convergence and effects of message errors. Journal of Machine Learning Research, 6:905-936.
  7. Wainwright, M. J. and Jordan, M. I. (2007). Graphical Models, Exponential Families, and Variational Inference. Foundations and Trends in Machine Learning, 1(1- 2):1-305.
Download


Paper Citation


in Harvard Style

Piatkowski N., Lee S. and Morik K. (2014). The Integer Approximation of Undirected Graphical Models . In Proceedings of the 3rd International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-758-018-5, pages 296-304. DOI: 10.5220/0004831202960304


in Bibtex Style

@conference{icpram14,
author={Nico Piatkowski and Sangkyun Lee and Katharina Morik},
title={The Integer Approximation of Undirected Graphical Models},
booktitle={Proceedings of the 3rd International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2014},
pages={296-304},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004831202960304},
isbn={978-989-758-018-5},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 3rd International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - The Integer Approximation of Undirected Graphical Models
SN - 978-989-758-018-5
AU - Piatkowski N.
AU - Lee S.
AU - Morik K.
PY - 2014
SP - 296
EP - 304
DO - 10.5220/0004831202960304