FEASIBLE CONTROL OF COMPLEX SYSTEMS USING AUTOMATIC LEARNING

Alejandro Agostini, Enric Celaya

2005

Abstract

Robotic applications often involve dealing with complex dynamic systems. In these cases coping with control requirements with conventional techniques is hard to achieve and a big effort has to be done in the design and tuning of the control system. An alternative to conventional control techniques is the use of automatic learning systems that could learn control policies automatically, by means of the experience. But the amount of experience required in complex problems is intractable unless some generalization is performed. Many learning techniques have been proposed to deal with this challenge but the applicability of them in a complex control task is still difficult because of their bad learning convergence or insufficient generalization. In this work a new learning technique, that exploits a kind of generalization called categorization, is used in a complex control task. The results obtained show that it is possible to learn, in short time and with good convergence, a control policy that outperforms a classical PID control tuned for the specific task of controlling a manipulator with high inertia and variable load.

References

  1. Agostini, A., Celaya, E., 2004a. Learning in Complex Environments with Feature-Based Categorization. In Proc. of 8th Conference on Intelligent Autonomous Systems. Amsterdam, The Netherland, pp. 446-455.
  2. Agostini, A., Celaya, E., 2004b. Trajectory Tracking Control of a Rotational Joint using Feature Based Categorization Learning. In Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems. Sendai, Japan, pp. 3489-3494.
  3. Blom, G., 1989. Probability and Statistics: Theory and Applications. The book. Springer-Verlag.
  4. Craig, J., 1989. Introduction to Robotics. The book, 2nd Ed. Addison-Wesley Publishing Company.
  5. Grossberg, S., 1982. A Psychophysiological Theory of Reinforcement, Drive, Motivation and Attention. Journal of Theoretical Neurobiology, 1, 286 369.
  6. Martins-Filho, L., Silvino, j., Presende, P., Assunçao, T., 2003. Control of robotic leg joints - comparing PD and sliding mode approaches. In Proc. of the Sixth International Conference on Climbing and Walking Robots (CLAWAR2003). Catania, Italy, pp.147-153.
  7. Ogata, K., 2002. Modern Control Engineering. The book. 4th ed., Prentice Hall, New Jersey, United State.
  8. Porta, J. M. and Celaya, E., 2000. Learning in Categorizable Environments. In Proc. of the Sixth International Conference on the Simulation of Adaptive Behavior (SAB2000). Paris, pp.343-352.
  9. Smart, W. and Kaelbling, L., 2000. Practical reinforcement learning in continuous spaces. In Proc. of the Seventeenth International Conference on Machine Learning (ICML).
  10. Sutton, R. and Barto, 1998. Reinforcement Learning. An Introduction. The book. MIT Press.
  11. Thrun, S. and Schwartz, A., 1993. Issues in Using Function Approximation for Reinforcement Learning. In Proc. of the Connectionist Models Summer School. Hillsdale, NJ, pp. 255-263.
  12. Tsitsikilis, J. and Van Roy, B., 1997. An Analysis of Temporal Difference Learning with Function Approximation. IEEE Transactions on Automatic Control, 42(5):674--690.
  13. Watkins, C., Dayan, P., 1992. Q-Learning. Machine Learning, 8:279-292.
Download


Paper Citation


in Harvard Style

Agostini A. and Celaya E. (2005). FEASIBLE CONTROL OF COMPLEX SYSTEMS USING AUTOMATIC LEARNING . In Proceedings of the Second International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO, ISBN 972-8865-29-5, pages 284-287. DOI: 10.5220/0001174202840287


in Bibtex Style

@conference{icinco05,
author={Alejandro Agostini and Enric Celaya},
title={FEASIBLE CONTROL OF COMPLEX SYSTEMS USING AUTOMATIC LEARNING},
booktitle={Proceedings of the Second International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO,},
year={2005},
pages={284-287},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0001174202840287},
isbn={972-8865-29-5},
}


in EndNote Style

TY - CONF
JO - Proceedings of the Second International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO,
TI - FEASIBLE CONTROL OF COMPLEX SYSTEMS USING AUTOMATIC LEARNING
SN - 972-8865-29-5
AU - Agostini A.
AU - Celaya E.
PY - 2005
SP - 284
EP - 287
DO - 10.5220/0001174202840287