In this paper a method for online solution of the Hamilton-Jacobi–Bellman (HJB) equation is proposed. The method is utilized to design an optimal controller for continuous-time nonlinear systems. The main concept in this approach is using experiences to reinforce the controller, which is called Reinforcement Learning (RL). The online solution is based on the actor-critic (AC) structure where two Neural Networks (NNs) approximately solve the HJB equation. Optimal control and optimal value function are approximated by the actor and the critic, respectively. Then, employing gradient descent algorithm, improves accuracy of the approximation. Since some items like friction and damping are difficult to model and calculate, a neural-robust identifier is used in conjunction with the AC to approximate drift dynamics. Finally the Actor-Critic-Identifier (ACI) structure is proposed to solve the HJB equation online without a prior knowledge of drift dynamics. The closed-loop stability of the overall system is assured by the Lyapunov theory employing the direct method. Then the effectiveness of the proposed method is illustrated by experiment for DC motor and simulation for a nonlinear system. Results indicate satisfactory performance of the proposed method to solve the Hamilton-Jacobi-Bellman equation.