PROCESS CONTROL USING CONTROLLED FINITE MARKOV CHAINS WITH AN APPLICATION TO A MULTIVARIABLE HYBRID PLANT

Enso Ikonen

2007

Abstract

Predictive and optimal process control using finite Markov chains is considered. A basic procedure is outlined, consisting of discretization of plant input and state spaces; conversion of a (a priori) plant model into a set of finite state probability transition maps; specification of immediate costs for state-action pairs; computation of an optimal or a predictive control policy; and, analysis of the closed-loop system behavior. An application, using a MATLAB toolbox developed for MDP-based process control design, illustrates the approach in the control of a multivariable plant with both discrete and continuous action variables. For problems of size of practical significance (thousands of states), computations can be performed on a standard office PC. The aim of the work is to provide a basic framework for examination of nonlinear control, emphasizing in on-line learning from uncertain data.

References

  1. A°kesson, B. M., Nikus, M. J., and Toivonen, H. T. (2006). Explicit model predictive control of a hybrid system using support vector machines. In Proceedings of the 1st IFAC Workshop on Applications of Large Scale Industrial Systems (ALSIS'06), Helsinki-Stockholm, Finland-Sweden.
  2. Häggström, O. (2002). Finite Markov Chains and Algorithmic Applications. Cambridge University Press, Cambridge.
  3. Hsu, C. S. (1987). Cell-to-Cell Mapping - A Method of Global Analysis for Nonlinear Systems. SpringerVerlag, New York.
  4. Ikonen, E. (2004). Learning predictive control using probabilistic models. In IFAC Workshop on Advanced Fuzzy/Neural Control (AFNC'04), Oulu, Finland.
  5. Ikonen, E. and Najim, K. (2002). Advanced Process Identification and Control. Marcel Dekker, New York.
  6. Kaelbling, L. P., Littman, M. L., and Moore, A. W. (1996). Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4:237-285.
  7. Lee, J. M. and Lee, J. H. (2004). Approximate dynamic programming strategies and their applicability for process control: A review and future directions. International Journal of Control, Automation, and Systems, 2(3):263-278.
  8. Lunze, J. (1998). On the Markov property of quantised state measurement sequences. Automatica, 34(11):1439- 1444.
  9. Lunze, J., Nixdorf, B., and Richter, H. (2001). Process supervision by means of a hybrid model. Journal of Process Control, 11:89-104.
  10. Najim, K., Ikonen, E., and Ait-Kadi, D. (2004). Stochastic Processes - Estimation, Optimization and Analysis. Kogan Page Science, London.
  11. Negenborn, R. R., De Schutter, B., Wiering, M. A., and Hellendoorn, H. (2005). Learning-based model predictive control for Markov decision processes. In 16th IFAC World Congress.
  12. Poznyak, A. S., Najim, K., and Gómez-Ramírez, E. (2000). Self-Learning Control of Finite Markov Chains. Marcel Dekker, New York.
  13. Puterman, M. L. (1994). Markov Decision Processes - Discrete Stochastic Dynamic Programming. Wiley et Sons, New York.
Download


Paper Citation


in Harvard Style

Ikonen E. (2007). PROCESS CONTROL USING CONTROLLED FINITE MARKOV CHAINS WITH AN APPLICATION TO A MULTIVARIABLE HYBRID PLANT . In Proceedings of the Fourth International Conference on Informatics in Control, Automation and Robotics - Volume 3: ICINCO, ISBN 978-972-8865-84-9, pages 78-85. DOI: 10.5220/0001615300780085


in Bibtex Style

@conference{icinco07,
author={Enso Ikonen},
title={PROCESS CONTROL USING CONTROLLED FINITE MARKOV CHAINS WITH AN APPLICATION TO A MULTIVARIABLE HYBRID PLANT},
booktitle={Proceedings of the Fourth International Conference on Informatics in Control, Automation and Robotics - Volume 3: ICINCO,},
year={2007},
pages={78-85},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0001615300780085},
isbn={978-972-8865-84-9},
}


in EndNote Style

TY - CONF
JO - Proceedings of the Fourth International Conference on Informatics in Control, Automation and Robotics - Volume 3: ICINCO,
TI - PROCESS CONTROL USING CONTROLLED FINITE MARKOV CHAINS WITH AN APPLICATION TO A MULTIVARIABLE HYBRID PLANT
SN - 978-972-8865-84-9
AU - Ikonen E.
PY - 2007
SP - 78
EP - 85
DO - 10.5220/0001615300780085