Authors:
Randa Herzallah
1
and
David Lowe
2
Affiliations:
1
Faculty of Engineering Technology, Al-Balqa’ Applied University, Jordan
;
2
NCRG, Aston University, United Kingdom
Keyword(s):
Adaptive critic methods, functional uncertainty, stochastic control.
Related
Ontology
Subjects/Areas/Topics:
Informatics in Control, Automation and Robotics
;
Intelligent Control Systems and Optimization
;
Machine Learning in Control Applications
Abstract:
Adaptive critic methods have common roots as generalizations of dynamic programming for neural reinforcement learning approaches. Since they approximate the dynamic programming solutions, they are potentially suitable for learning in noisy, nonlinear and nonstationary environments. In this study, a novel probabilistic dual heuristic programming (DHP) based adaptive critic controller is proposed. Distinct to current approaches, the proposed probabilistic (DHP) adaptive critic method takes uncertainties of forward model and inverse controller into consideration. Therefore, it is suitable for deterministic and stochastic control problems characterized by functional uncertainty. Theoretical development of the proposed method is validated by analytically evaluating the correct value of the cost function which satisfies the Bellman equation in a linear quadratic control problem. The target value of the critic network is then calculated and shown to be equal to the analytically derived corr
ect value.
(More)