Adaptive dynamic programming-based design of integrated neural network structure

What is it about?

Solving cooperative problems for multi-agent systems, in which the agent's artificial behaviors are similar to naturally biological behaviors of agents in practice, is a major challenge. The problems become more complex if the controlled agents are multi-input and multi-output (MIMO) nonlinear systems lacking knowledge of internal system dynamics and affected by external disturbances. In this paper, firstly, based on adaptive dynamic programming, three neural networks (NNs) (actor/disturber/critic) of control schemes for two-player games are integrated into the structure with only one NN, known as integrated NN (INN), with the aim of reducing computational complexity and waste of resources. Secondly, an INN weight update law and an online control algorithm, which updates parameters in one iterative step, are designed to find H1 optimal cooperative control solutions. With the aid of Lyapunov theory, we prove that the INN weight approximation errors and the cooperative tracking errors are uniformly ultimately bounded (UUB), and the system parameters converge to the approximately optimal values. Finally, two simulation studies, one of which is compared to three-NN structures in existing literature, are carried out to show the effectiveness of the proposed INN structure.

Featured Image

Why is it important?

1. Design an integrated neural network (INN) structure in contrast to the existing work in [4]. To the best of our knowledge, this paper may be the first work that three NNs in the ADP method are integrated into only one for the cooperative problem of the multiple MIMO nonlinear systems. This integration aims to reduce computational complexity and resources. 2. Design an INN weight update law, in which the knowledge of internal system dynamics is not required, and an online H-inf optimal cooperative control algorithm, in which the INN weight parameters and the parameters of the control and disturbance laws are simultaneously and continuously updated in one iterative step. 3. Prove that, with the aid of Lyapunov theory, cooperative tracking errors of the closed-loop system and the INN weight approximation errors are UUB, and the value function, the H-inf optimal cooperative control law and the worst disturbance law converge to approximately optimal values. 4. The effectiveness of our method is shown by comparing the simulation results of both INN and existing three-NN structures