A novel actor-critic-identifier architecture for approximate optimal control of uncertain nonlinear systems. (English) Zbl 1257.93055

Summary: An online adaptive reinforcement learning-based solution is developed for the infinite-horizon optimal control problem for continuous-time uncertain nonlinear systems. A novel Actor-Critic-Identifier (ACI) is proposed to approximate the Hamilton-Jacobi-Bellman equation using three Neural Network (NN) structures-actor and critic NNs approximate the optimal control and the optimal value function, respectively, and a robust dynamic neural network identifier asymptotically approximates the uncertain system dynamics. An advantage of using the ACI architecture is that learning by the actor, critic, and identifier is continuous and simultaneous, without requiring knowledge of system drift dynamics. Convergence of the algorithm is analyzed using Lyapunov-based adaptive control methods. A persistence of excitation condition is required to guarantee exponential convergence to a bounded region in the neighborhood of the optimal control and Uniformly Ultimately Bounded (UUB) stability of the closed-loop system. Simulation results demonstrate the performance of the actor-critic-identifier method for approximate optimal control.


93C40 Adaptive control/observation systems
68T05 Learning and adaptive systems in artificial intelligence
49L20 Dynamic programming in optimal control and differential games
93D20 Asymptotic stability in control theory
Full Text: DOI