Volume 16, Issue 10 (1-2017)
Modares Mechanical Engineering 2017, 16(10): 241-252 |
Back to browse issues page
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
Mojoodi A, Naraghi M, Moradi M. Online solution of the Hamilton–Jacobi–Bellman equation for nonlinear systems with unknown drift dynamics using neural network. Modares Mechanical Engineering 2017; 16 (10) :241-252
URL: http://mme.modares.ac.ir/article-15-9392-en.html
URL: http://mme.modares.ac.ir/article-15-9392-en.html
1- Mechanical Engineering Department, Amirkabir University of Technology (Tehran Polytechnic)
2- Associate Professor of Mechanical Engineering Department, Amirkabir University of Technology (Tehran Polytechnic)
2- Associate Professor of Mechanical Engineering Department, Amirkabir University of Technology (Tehran Polytechnic)
Abstract: (7049 Views)
In this paper a method for online solution of the Hamilton-Jacobi–Bellman (HJB) equation is proposed. The method is utilized to design an optimal controller for continuous-time nonlinear systems. The main concept in this approach is using experiences to reinforce the controller, which is called Reinforcement Learning (RL). The online solution is based on the actor-critic (AC) structure where two Neural Networks (NNs) approximately solve the HJB equation. Optimal control and optimal value function are approximated by the actor and the critic, respectively. Then, employing gradient descent algorithm, improves accuracy of the approximation. Since some items like friction and damping are difficult to model and calculate, a neural-robust identifier is used in conjunction with the AC to approximate drift dynamics. Finally the Actor-Critic-Identifier (ACI) structure is proposed to solve the HJB equation online without a prior knowledge of drift dynamics. The closed-loop stability of the overall system is assured by the Lyapunov theory employing the direct method. Then the effectiveness of the proposed method is illustrated by experiment for DC motor and simulation for a nonlinear system. Results indicate satisfactory performance of the proposed method to solve the Hamilton-Jacobi-Bellman equation.
Article Type: Research Article |
Subject:
Control
Received: 2016/06/21 | Accepted: 2016/08/27 | Published: 2016/10/15
Received: 2016/06/21 | Accepted: 2016/08/27 | Published: 2016/10/15
| Rights and permissions | |
 |
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |