In this paper, the neural-network-based robust optimal control design for a class of uncertain nonlinear systems via adaptive dynamic programming approach is investigated. First, the robust controller of the original uncertain system is derived by adding a feedback gain to the optimal controller of the nominal system. It is also shown that this robust controller can achieve optimality under a specified cost function, which serves as the basic idea of the robust optimal control design. Then, a critic network is constructed to solve the Hamilton–Jacobi–Bellman equation corresponding to the nominal system, where an additional stabilizing term is introduced to verify the stability. The uniform ultimate boundedness of the closed-loop system is also proved by using the Lyapunov approach. Moreover, the obtained results are extended to solve decentralized optimal control problem of continuous-time nonlinear interconnected large-scale systems. Finally, two simulation examples are presented to illustrate the effectiveness of the established control scheme.
In practical control systems, model uncertainties arise frequently and can severely degrade the closed-loop system performance. Hence, the problem of designing robust controller for nonlinear systems with uncertainties has drawn considerable attention in recent literature [43], [15] and [31]. Lin et al. [15] showed that the robust control problem can be solved by studying the optimal control problem of the corresponding nominal system, but the detailed procedure was not presented. In [31], the authors developed an iterative algorithm for online design of robust control for a class of continuous-time nonlinear systems. However, the optimality of the robust controller with respect to a specified cost function was not discussed. In [43], the authors addressed the problem of designing robust tracking controls for a class of uncertain nonholonomic systems actuated by brushed direct current motors, while the research was not related with the optimality.
The starting point of the obtained strategy of this paper is optimal control. The nonlinear optimal control problem always requires to solve the Hamilton–Jacobi–Bellman (HJB) equation. Though dynamic programming has been a conventional method in solving optimization and optimal control problems, it often suffers from the curse of dimensionality, which was primarily due to the backward-in-time approach. To avoid the difficulty, based on function approximators, such as neural networks, adaptive/approximate dynamic programming (ADP) was proposed by Werbos [35] as a method to solve optimal control problems forward-in-time. Recently, the study on ADP and related fields have gained much attention from various scholars [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [12], [13], [14], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [28], [29], [30], [32], [33], [34], [36], [37], [38], [40], [41], [42], [44], [45] and [46]. Lewis and Vrabie [13] stated that the ADP technique is closely related to the field of reinforcement learning. As is known to all, policy iteration is one of the basic algorithms of reinforcement learning. In addition, the initial admissible control is necessary when employing the policy iteration algorithm. However, in many situations, it is difficult to find the initial admissible control.
To the best of our knowledge, there are few results on robust optimal control of uncertain nonlinear systems based on ADP, not to mention the decentralized optimal control of large-scale systems. This is the motivation of our research. Actually, it is the first time that the robust optimal control scheme for a class of uncertain nonlinear systems via ADP technique and without using an initial admissible control is established. To begin with, the optimal controller of the nominal system is designed. It can be proved that the modification of optimal control law is in fact the robust controller of the original uncertain system, which also achieves optimality under the definition of a cost function. Then, a critic network is constructed for solving the HJB equation corresponding to the nominal system. In addition, inspired by the work of [5] and [24], an additional stabilizing term is introduced to verify the stability, which relaxes the need for an initial stabilizing control. The uniform ultimate boundedness (UUB) of the closed-loop system is also proved via the Lyapunov approach. Furthermore, the aforementioned results are extended to deal with the decentralized optimal control for a class of continuous-time nonlinear interconnected systems. At last, two simulation examples are given to show the effectiveness of the robust optimal control scheme.
A novel robust optimal control scheme for a class of uncertain nonlinear systems via ADP approach is developed in this paper. It is proved that the robust controller of the original uncertain system achieves optimality under a specified cost function. During the implementation process, a critic network is constructed to solve the HJB equation of the nominal system and an additional stabilizing term is introduced to verify the stability. The obtained results are also extended to design the decentralized optimal control for a class of nonlinear interconnected large-scale systems. Simulation studies verify the good control performance. In the future, we will focus on studying the robust and decentralized optimal control for nonlinear systems with unknown dynamics. In this sense, the requirement of system dynamics will be further reduced, which reflects the superiority of ADP technique in dealing with the optimal control problem under nonlinear, uncertain, and complex environment.
