Volume 16, Issue 10 (1-2017)                   Modares Mechanical Engineering 2017, 16(10): 241-252 | Back to browse issues page

XML Persian Abstract Print

Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:

Mojoodi A, Naraghi M, Moradi M. Online solution of the Hamilton–Jacobi–Bellman equation for nonlinear systems with unknown drift dynamics using neural network. Modares Mechanical Engineering 2017; 16 (10) :241-252
URL: http://mme.modares.ac.ir/article-15-9392-en.html
1- Mechanical Engineering Department, Amirkabir University of Technology (Tehran Polytechnic)
2- Associate Professor of Mechanical Engineering Department, Amirkabir University of Technology (Tehran Polytechnic)
Abstract:   (7173 Views)
In this paper a method for online solution of the Hamilton-Jacobi–Bellman (HJB) equation is proposed. The method is utilized to design an optimal controller for continuous-time nonlinear systems. The main concept in this approach is using experiences to reinforce the controller, which is called Reinforcement Learning (RL). The online solution is based on the actor-critic (AC) structure where two Neural Networks (NNs) approximately solve the HJB equation. Optimal control and optimal value function are approximated by the actor and the critic, respectively. Then, employing gradient descent algorithm, improves accuracy of the approximation. Since some items like friction and damping are difficult to model and calculate, a neural-robust identifier is used in conjunction with the AC to approximate drift dynamics. Finally the Actor-Critic-Identifier (ACI) structure is proposed to solve the HJB equation online without a prior knowledge of drift dynamics. The closed-loop stability of the overall system is assured by the Lyapunov theory employing the direct method. Then the effectiveness of the proposed method is illustrated by experiment for DC motor and simulation for a nonlinear system. Results indicate satisfactory performance of the proposed method to solve the Hamilton-Jacobi-Bellman equation.
Full-Text [PDF 1298 kb]   (5984 Downloads)    
Article Type: Research Article | Subject: Control
Received: 2016/06/21 | Accepted: 2016/08/27 | Published: 2016/10/15

Add your comments about this article : Your username or Email:

Rights and permissions
Creative Commons License This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.