BATCH REINFORCEMENT LEARNING - An Application to a Controllable Semi-active Suspension System

Simone Tognetti, Marcello Restelli, Sergio M. Savaresi, Cristiano Spelta

2009

Abstract

The design problem of optimal comfort-oriented semi-active suspension has been addressed with different standard techniques which failed to come out with an optimal strategy because the system is hard non-linear and the solution is too complex to be found analytically. In this work, we aimed at solving such complex problem by applying Batch Reinforcement Learning (BRL), that is an artificial intelligence technique that approximates the solution of optimal control problems without knowing the system dynamics. Recently, a quasi optimal strategy for semi-active suspension has been designed and proposed: the Mixed SH-ADD algorithm, which the strategy designed in this paper is compared to. We show that an accurately tuned BRL provides a policy able to guarantee the overall best performance.

References

  1. Ahmadian, M., Reichert, B. A., and Song, X. (2001). System non-linearities induced by skyhook dampers. Shock and Vibration, 8(2):95-104.
  2. Antos, A., Munos, R., and Szepesvari, C. (2008). Fitted q-iteration in continuous action-space mdps. In Platt, J., Koller, D., Singer, Y., and Roweis, S., editors, Advances in Neural Information Processing Systems 20, pages 9-16. MIT Press, Cambridge, MA.
  3. Ernst, D., Geurts, P., Wehenkel, L., and Littman, L. (2005). Tree-based batch mode reinforcement learning. Journal of Machine Learning Research, 6:503-556.
  4. Geurts, P., Ernst, D., and Wehenkel, L. (2006). Extremely randomized trees. Machine Learning, 63(1):3-42.
  5. Guardabassi, G. and Savaresi, S. (2001). Approximate linearization via feedback - an overview. Survey paper on Automatica, 27:1-15.
  6. Hrovat, D. (1997). Survey of advanced suspension developments and related optimal control applications. Automatica(Oxford), 33(10):1781-1817.
  7. Kaelbling, L. P., Littman, M. L., and Moore, A. W. (1996). Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4:237-285.
  8. Karnopp, D. and Crosby, M. (1974). System for Controlling the Transmission of Energy Between Spaced Members. US Patent 3,807,678.
  9. Riedmiller, M. (2005). Neural fitted q iteration - first experiences with a data efficient neural reinforcement learning method. In ECML, pages 317-328.
  10. Sammier, D., Sename, O., and Dugard, L. (2003). Skyhook and H8 Control of Semi-active Suspensions: Some Practical Aspects. Vehicle System Dynamics, 39(4):279-308.
  11. Savaresi, S., Silani, E., and Bittanti, S. (2005). Acceleration-Driven-Damper (ADD): An Optimal Control Algorithm For Comfort-Oriented Semiactive Suspensions. Journal of Dynamic Systems, Measurement, and Control, 127:218.
  12. Savaresi, S. and Spelta, C. (2007). Mixed Sky-Hook and ADD: Approaching the Filtering Limits of a SemiActive Suspension. Journal of Dynamic Systems, Measurement, and Control, 129:382.
  13. Savaresi, S. and Spelta, C. (2008). A single-sensor control strategy for semi-active suspensions. To Appear, -:-.
  14. Silani, E., Savaresi, S., Bittanti, S., Visconti, A., and Farachi, F. (2002). The Concept of PerformanceOriented Yaw-Control Systems: Vehicle Model and Analysis. SAE Transactions, Journal of Passenger Cars - Mechanical Systems, 111(6):1808-1818. ISBN No.0-7680-1290-2,.
  15. Sutton, R. and Barto, A. (1998). Reinforcement Learning: An Introduction. MIT Press.
  16. Valasek, M., Kortum, W., Sika, Z., Magdolen, L., and Vaculin, O. (1998). Development of semi-active road-friendly truck suspensions. Control Engineering Practice, 6:735-744.
  17. Watkins, C. (1989). Learining from Delayed Rewards. PhD thesis, Cambridge University, Cambridge,England.
  18. Williams, R. (1997). Automotive active suspensions Part 1: basic principles. Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering, 211(6):415-426.
Download


Paper Citation


in Harvard Style

Tognetti S., Restelli M., M. Savaresi S. and Spelta C. (2009). BATCH REINFORCEMENT LEARNING - An Application to a Controllable Semi-active Suspension System . In Proceedings of the 6th International Conference on Informatics in Control, Automation and Robotics - Volume 3: ICINCO, ISBN 978-989-8111-99-9, pages 228-233. DOI: 10.5220/0002210302280233


in Bibtex Style

@conference{icinco09,
author={Simone Tognetti and Marcello Restelli and Sergio M. Savaresi and Cristiano Spelta},
title={BATCH REINFORCEMENT LEARNING - An Application to a Controllable Semi-active Suspension System},
booktitle={Proceedings of the 6th International Conference on Informatics in Control, Automation and Robotics - Volume 3: ICINCO,},
year={2009},
pages={228-233},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0002210302280233},
isbn={978-989-8111-99-9},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 6th International Conference on Informatics in Control, Automation and Robotics - Volume 3: ICINCO,
TI - BATCH REINFORCEMENT LEARNING - An Application to a Controllable Semi-active Suspension System
SN - 978-989-8111-99-9
AU - Tognetti S.
AU - Restelli M.
AU - M. Savaresi S.
AU - Spelta C.
PY - 2009
SP - 228
EP - 233
DO - 10.5220/0002210302280233