CMAC STRUCTURE OPTIMIZATION WITH Q-LEARNING APPROACH AND ITS APPLICATION

Weiwei Yu, Kurosh Madani, Christophe Sabourin

2011

Abstract

Comparing with other neural networks based models, CMAC is successfully applied on many nonlinear control systems because of its computational speed and learning ability. However, for high-dimensional input cases in real application, we often have to make our choice between learning accuracy and memory size. This paper discusses how both the number of layer and step quantization influence the approximation quality of CMAC. By experimental enquiry, it is shown that it is possible to decrease the memory size without losing the approximation quality by selecting the adaptive structural parameters. Based on Q-learning approach, the CMAC structural parameters can be optimized automatically without increasing the complexity of its structure. The choice of this optimized CMAC structure can achieve a tradeoff between the learning accuracy and finite memory size. At last, the application of this Q-learning based CMAC structure optimization approach on the joint angle tracking problem for biped robot is presented.

References

  1. J. S. Albus, 1975. A new approach to manipulator control: The cerebellar model articulation controller (CMAC). Transactions of the ASME: Journal of Dynamic Systems, Measurement, and Control, 9:220-227.
  2. Hung-Ching Lu, Ming-Feng Yeh, Jui-Chi Chang, 2006. CMAC study with adaptive quantization. IEEE Int. Conf. on Systems, Man, and Cybernetics, pp.2596- 2601.
  3. S. D. Teddy, E. M.-K. Lai, C. Quek, 2007. Hierarchically clustered adaptive quantization CMAC and its learning convergence. IEEE Trans. on Neural Networks, 18(6):1658-1682.
  4. A. Menozzi, M. Chow, 1997. On the training of a multiresolution CMAC neural network. Proc. Int. Conf. Ind. Electron. Control Instrum. 3:1130-1135.
  5. Chih-Min Lin, Te-Yu Chen, 2009. Self-organizing CMAC control for a class of MIMO uncertain nonlinear systems. IEEE Trans. on Neural Networks, 20(9):1377-1384.
  6. M. N. Nguyen, D. Shi, C. Quek, 2005. Self-organizing Gaussian fuzzy CMAC with truth value restriction. Proc. IEEE ICITA, pp.185-190, Sydney, Australia.
  7. Daming Shi, Minh Nhut Nguyen, Suiping Zhou, Guisheng Yin, 2010. Fuzzy CMAC with incremental Bayesian Ying-Yang learning and dynamic rule construction, IEEE Trans. on Systems, Man and Cybernetics, 40(2):548-552.
  8. C. Watkins, P. Dayan, 1992. Q-learning. Machine Learning, 8:279-292.
  9. C. Sabourin, K. Madani, Weiwei Yu, Jie Yan, 2008. Obstacle Avoidance Strategy for Biped Robot Based on Fuzzy Q-Learning Approach, International Conference on Automatic Control & Robotics, Portugal.
Download


Paper Citation


in Harvard Style

Yu W., Madani K. and Sabourin C. (2011). CMAC STRUCTURE OPTIMIZATION WITH Q-LEARNING APPROACH AND ITS APPLICATION . In Proceedings of the International Conference on Neural Computation Theory and Applications - Volume 1: NCTA, (IJCCI 2011) ISBN 978-989-8425-84-3, pages 283-288. DOI: 10.5220/0003694102830288


in Bibtex Style

@conference{ncta11,
author={Weiwei Yu and Kurosh Madani and Christophe Sabourin},
title={CMAC STRUCTURE OPTIMIZATION WITH Q-LEARNING APPROACH AND ITS APPLICATION},
booktitle={Proceedings of the International Conference on Neural Computation Theory and Applications - Volume 1: NCTA, (IJCCI 2011)},
year={2011},
pages={283-288},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003694102830288},
isbn={978-989-8425-84-3},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Neural Computation Theory and Applications - Volume 1: NCTA, (IJCCI 2011)
TI - CMAC STRUCTURE OPTIMIZATION WITH Q-LEARNING APPROACH AND ITS APPLICATION
SN - 978-989-8425-84-3
AU - Yu W.
AU - Madani K.
AU - Sabourin C.
PY - 2011
SP - 283
EP - 288
DO - 10.5220/0003694102830288