How New Information Criteria WAIC and WBIC Worked for MLP Model Selection

Seiya Satoh, Ryohei Nakano

2017

Abstract

The present paper evaluates newly invented information criteria for singular models. Well-known criteria such as AIC and BIC are valid for regular statistical models, but their validness for singular models is not guaranteed. Statistical models such as multilayer perceptrons (MLPs), RBFs, HMMs are singular models. Recently WAIC and WBIC have been proposed as new information criteria for singular models. They are developed on a strict mathematical basis, and need empirical evaluation. This paper experimentally evaluates how WAIC and WBIC work for MLP model selection using conventional and new learning methods.

References

  1. Akaike, H. (1974). A new look at the statistical model identification. IEEE Trans. on Automatic Control, AC19:716-723.
  2. Ando, T. (2007). Bayesian predictive information criterion for the evaluation of hierarchical Bayesian and empirical Bayes models. Biometrika, 94:443-458.
  3. Neal, R. (1996). Bayesian learning for neural networks. Springer.
  4. Saito, K. and Nakano, R. (1997). Partial BFGS update and efficient step-length calculation for three-layer neural networks. Neural Comput., 9(1):239-257.
  5. Satoh, S. and Nakano, R. (2012). Eigen vector descent and line search for multilayer perceptron. In IAENG Int. Conf. on Artificial Intelligence & Applications (ICAIA'12), volume 1, pages 1-6.
  6. Satoh, S. and Nakano, R. (2013a). Fast and stable learning utilizing singular regions of multilayer perceptron. Neural Processing Letters, 38(2):99-115.
  7. Satoh, S. and Nakano, R. (2013b). Multilayer perceptron learning utilizing singular regions and search pruning. In Proc. Int. Conf. on Machine Learning and Data Analysis, pages 790-795.
  8. Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6:461-464.
  9. Watanabe, S. (2009). Algebraic geometry and statistical learning theory. Cambridge University Press, Cambridge.
  10. Watanabe, S. (2010). Equations of states in singular statistical estimation. Neural Networks, 23:20-34.
  11. Watanabe, S. (2013). A widely applicable Bayesian information criterion. Journal of Machine Learning Research, 14:867-897.
Download


Paper Citation


in Harvard Style

Satoh S. and Nakano R. (2017). How New Information Criteria WAIC and WBIC Worked for MLP Model Selection . In Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-758-222-6, pages 105-111. DOI: 10.5220/0006120301050111


in Bibtex Style

@conference{icpram17,
author={Seiya Satoh and Ryohei Nakano},
title={How New Information Criteria WAIC and WBIC Worked for MLP Model Selection},
booktitle={Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2017},
pages={105-111},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006120301050111},
isbn={978-989-758-222-6},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - How New Information Criteria WAIC and WBIC Worked for MLP Model Selection
SN - 978-989-758-222-6
AU - Satoh S.
AU - Nakano R.
PY - 2017
SP - 105
EP - 111
DO - 10.5220/0006120301050111