The aim of this paper is to extend the dynamic programming (DP) approach to multi-model optimal control problems (OCPs). We deal with robust optimization of multi-model control systems and are particularly interested in the Hamilton–Jacobi–Bellman (HJB) equation for the above class of problems. In this paper, we study a variant of the HJB for multi-model OCPs and examine the natural relationship between the Bellman DP techniques and the Robust Maximum Principle (MP). Moreover, we describe how to carry out the practical calculations in the context of multi-model LQ-problems and derive the associated Riccati-type equation.
The theory of OCPs governed by ordinary differential equations has been well established since the middle of the 20th century; see e.g., [1], [2], [3], [4], [5] and [6] and the references therein. For a classical OCP, the main tools toward the construction of optimal trajectories, and then optimal synthesis, are the celebrated Pontryagin MP and the Bellman DP. Recently robust optimization problems for multi-model control systems have attracted a lot of attention; thus both theoretical results and applications were developed, (see [7], [8], [9], [10] and [11]). OCPs for multi-model dynamical systems arise in the control of mechanical multibody systems, electrical circuits and heterogeneous systems, where different models are coupled together. The majority of applied OCPs are problems with incomplete information on the model structure or parameters. The multi-model control systems provide useful theoretical models for some classes of dynamical systems with the above-mentioned types of uncertainties. In this case one of the most efficient approaches to the optimal design of such systems is the robust optimization technique. Optimal robust control strategies based on the minimax algorithms have found a wide use in the design of complex control systems. Robust MP proposed by Boltyanski and Poznyak (see e.g., [7], [8], [9], [10] and [11]) is the basic analytical result for studying OCPs with multi-model controlled plants. On the other hand, the Bellman DP techniques are not far enough advanced for multi-model OCPs.
The purpose of this paper is to apply the classic DP techniques to a class of multi-model OCPs. First, we verify the Bellman principle of optimality for the class of problems under consideration. Second, we derive a (robust) version of the HJB equation. It should be stressed that our main result deals with a finite parametric set involved into a model description. We also apply the HJB equation to a multi-model LQ-problem (see [9] and [10]) and derive a parametric Riccati equation for the linear-quadratic case. Moreover, the obtained theoretical facts are considered in comparison with the corresponding theorem resulting from the application of the Robust MP to multi-model LQ-problems [10]. In such a manner we establish the natural relationship between DP and the Robust MP for the given class of LQ-problems (see e.g., [12]).
The remainder of our paper is organized as follows. Section 2 contains a problem formulation, some basic concepts and preliminary results. Section 3 is devoted to the main result of this paper, namely, to a variant of the HJB equation for multi-model OCPs. Moreover, we also deal with the corresponding verification techniques. In Section 4 we apply our theoretical results to the multi-model linear quadratic problems and deduce a Riccati-formalism similar to the classic LQ-theory. Section 5 summarizes the paper.
This paper deals with multi-model OCPs in the context of the Bellman DP. We derive a robust variant of the HJB equation and formulate the corresponding verification theorem. Moreover, we establish the relationship between the Robust MP and the obtained variant of the HJB equation for multi-model OCPs. In particular, for the LQ-type OCPs we deduce the Riccati-formalism similar to the classic LQ-theory, and show that the results obtained using the robust HJB equation coincide with consequences of the applications of the Robust MP to the multi-model LQ-problems. The main results presented in this paper are based on the assumption that the value function and the verification function are smooth. It is well known that this assumption does not necessarily hold and that the viscosity solution theory provides an excellent framework to deal with the above problem [19] and [20]. Evidently, a generalization of the corresponding classic concepts to the multi-model OCPs in the framework of the HJB equation is a challenging problem Finally, note that the approaches and results presented in this paper can also be extended to the multi-model OCPs with target/state constraints and to some classes of hybrid and switched systems (see e.g., [21]). Moreover, all the above-mentioned DP techniques are developed for multi-model OCPs with a finite parametric set. It seems to be possible to generalize these techniques for multi-model systems with infinite parametric sets and measured spaces as uncertainty models (see [8]).
