MULTILAYER SPLINE-BASED FUZZY NEURAL NETWORK WITH DETERMINISTIC INITIALIZATION

Vitaliy Kolodyazhniy

2011

Abstract

A multilayer spline-based fuzzy neural network (MS-FNN) is proposed. It is based on the concept of multilayer perceptron (MLP) with B-spline receptive field functions (Spline Net). In this paper, B-splines are considered in the framework of fuzzy set theory as membership functions such that the entire network can be represented in form of fuzzy rules. MS-FNN does not rely on tensor-product construction of basis functions. Instead, it is constructed as a multilayered superposition of univariate synaptic functions and thus avoids the curse of dimensionality similarly to MLP, yet with improved local properties. Additionally, a fully deterministic initialization procedure based on principal component analysis is proposed for MS-FNN, in contrast to the usual random initialization of multilayer networks. Excellent performance of MS-FNN with one and two hidden layers, different activation functions, and B-splines of different orders is demonstrated for time series prediction and classification problems.

References

  1. Haykin, S., 1998. Neural Networks: A Comprehensive Foundation, Prentice Hall, Upper Saddle River, 2nd edition.
  2. Lane S. H., Flax, M. G., Handelman, D. A., Gelfand, J. J., 1990. Multi-Layer Perceptrons with B-Spline Receptive Field Functions. In Advances in Neural Information Processing Systems, R. P. Lippman, J. Moody, D. S. Touretzky (Eds.), Vol. 3,. Morgan Kaufman, San Francisco, pp. 684-692.
  3. Xiang, C., Ding, S. Q., Lee, T.H., 2005. Geometrical Interpretation and Architecture Selection of MLP. IEEE Transactions on Neural Networks, 16, 84-96.
  4. Barron, A. R., 1992. Neural Net Approximation, In Proceedings of the 7th Yale Workshop Adaptive and Learning Systems, New Haven, CT, 1992, pp. 69-72.
  5. Barron, A. R., 1993. Universal Approximation Bounds for Superpositions of a Sigmoidal Function. IEEE Transactions on Information Theory, 39, 930-945.
  6. Nguyen, D., Widrow, B., 1990. Improving the Learning Speed of 2-layer Neural Networks by Choosing Initial Values of the Adaptive Weights. In Proceedings of the International Joint Conference on Neural Networks, vol. 3, 1990, pp. 21-26.
  7. Lehtokangas, M., Saarinen, J., Kaski, K., Huuhtanen, P., 1995. Initializing Weights of a Multilayer Perceptron Network by Using the Orthogonal Least Squares Algorithm. Neural Computation, 7, 982-999.
  8. Erdogmus, D., Fontenla-Romero, O., Principe, J. C., Alonso-Betanzos, A., Castillo, E., 2005. Linear-LeastSquares Initialization of Multilayer Perceptrons Through Backpropagation of the Desired Response. IEEE Transactions on Neural Networks, 16, 325-337.
  9. Bodyanskiy, Ye., Gorshkov, Ye., Kolodyazhniy, V., Poyedyntseva, V., 2005. Neuro-fuzzy Kolmogorov's Network, Lecture Notes in Computer Science, W. Duch et al. (Eds.), Vol. 3697, Springer, Berlin, Heidelberg, New York, pp. 1-6.
  10. Kolodyazhniy, V., 2009. Spline-based Neuro-fuzzy Kolmogorov's Network for Time Series Prediction. In Proceedings of the 17th European Symposium on Artificial Neural Networks (ESANN 2009), Bruges, Belgium, 2009, pp. 153-158.
  11. Kolodyazhniy, V., Klawonn, F., Tschumitschew, K., 2007. A Neuro-fuzzy Model for Dimensionality Reduction and its Application. International Journal of Uncertainty, Fuzziness, and Knowledge-Based Systems, 15, 571-593.
  12. Zhang, J., Knoll, A., 1998. Constructing Fuzzy Controllers with B-Spline Models - Principles and Applications. International Journal of Intelligent Systems, 13, 257- 285.
  13. Yamakawa, T., Uchino, E., Miki, T., Kusanagi, H., 1992. A Neo Fuzzy Neuron and its Applications to System Identification and Prediction of the System Behavior, In Proceedings of the 2nd International Conference on Fuzzy Logic and Neural Networks (IIZUKA-92), Iizuka, Japan, 1992, pp. 477-483.
  14. Bishop, C. M., 1995. Neural Networks for Pattern Recognition, Clarendon Press, Oxford.
  15. Coelho, L. d. S., Pessôa, M. W., 2009. Nonlinear Identification Using a B-Spline Neural Network and Chaotic Immune Approaches. Mechanical Systems and Signal Processing, 23, 2418-2434.
  16. Wang, K., Lei, B., 2001. Using B-spline Neural Network to Extract Fuzzy Rules for a Centrifugal Pump Monitoring. Journal of Intelligent Manufacturing, 12, pp. 5-11.
  17. Cheng, K. W. E., Wang, H. Y., Sutanto, D., 2001. Adaptive Directive Neural Network Control for ThreePhase AC/DC PWM Converter, Proc. Inst. Electr. Eng.-Elect. Power Appl., vol. 148, no. 5, pp. 425- 430.
  18. Riedmiller, M., Braun, H., 1993. A Direct Adaptive Method for Faster Backpropagation Learning: The Rprop Algorithm. Proceedings of the IEEE International Conference on Neural Networks, 1993, pp. 586-591.
  19. Igel, C., Hüsken, M., 2003. Empirical Evaluation of the Improved Rprop Learning Algorithms. Neurocomputing, 50, 105-123.
  20. Wang, D., Zeng, X.-J., Keane, J. A., 2009. Intermediate Variable Normalization for Gradient Descent Learning for Hierarchical Fuzzy System. IEEE Transactions on Neural Networks, 17, 468-476.
  21. Mackey, M. C., Glass, L., 1977. Oscillation and Chaos in Physiological Control Systems. Science, 197, 287-289.
  22. Rumelhart, D. E., McClelland, J. L., 1986. Parallel Distributed Processing vol. 1, Cambridge, MIT press.
  23. Wilamowski, B.M., Yu H., 2010. Improved Computation for Levenberg-Marquardt Training. IEEE Transactions on Neural Networks, 21, 930-937.
  24. Wieland, A., 1988. Two spirals. http://www.boltz.cs.cmu.edu/benchmarks/twospirals.html. CMU Repository of Neural Network Benchmarks.
  25. Schwenk, H., Bengio, Y., 2000. Boosting Neural Networks. Neural Computation, 12, 1869-1887.
Download


Paper Citation


in Harvard Style

Kolodyazhniy V. (2011). MULTILAYER SPLINE-BASED FUZZY NEURAL NETWORK WITH DETERMINISTIC INITIALIZATION . In Proceedings of the International Conference on Evolutionary Computation Theory and Applications - Volume 1: FCTA, (IJCCI 2011) ISBN 978-989-8425-83-6, pages 475-481. DOI: 10.5220/0003652604750481


in Bibtex Style

@conference{fcta11,
author={Vitaliy Kolodyazhniy},
title={MULTILAYER SPLINE-BASED FUZZY NEURAL NETWORK WITH DETERMINISTIC INITIALIZATION},
booktitle={Proceedings of the International Conference on Evolutionary Computation Theory and Applications - Volume 1: FCTA, (IJCCI 2011)},
year={2011},
pages={475-481},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003652604750481},
isbn={978-989-8425-83-6},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Evolutionary Computation Theory and Applications - Volume 1: FCTA, (IJCCI 2011)
TI - MULTILAYER SPLINE-BASED FUZZY NEURAL NETWORK WITH DETERMINISTIC INITIALIZATION
SN - 978-989-8425-83-6
AU - Kolodyazhniy V.
PY - 2011
SP - 475
EP - 481
DO - 10.5220/0003652604750481