The input-constrained LQR problem is addressed in this paper; i.e., the problem of finding the optimal control law for a linear system such that a quadratic cost functional is minimised over a horizon of length N subject to the satisfaction of input constraints. A global solution (i.e., valid in the entire state space) for this problem, and for arbitrary horizon N, is derived analytically by using dynamic programming. The scalar input case is considered in this paper. Solutions to this problem (and to more general problems: state constraints, multiple inputs) have been reported recently in the literature, for example, approaches that use the geometric structure of the underlying quadratic programming problem and approaches that use multi-parametric quadratic programming techniques. The solution by dynamic programming proposed in the present paper coincides with the ones obtained by the aforementioned approaches. However, being derived using a different approach that exploits the dynamic nature of the constrained optimisation problem to obtain an analytical solution, the present result complements the previous methods and reveals additional insights into the intrinsic structure of the optimal solution.
The solution of constrained LQR problems has attracted considerable attention recently. This interest is, mainly, due to the fact that these problems constitute the core underlying optimisation problem that is solved, at each sampling time, by model predictive control algorithms (one of the most popular control methodologies used in industry at present). Of particular interest has been the derivation of explicit solutions that, through a characterisation of the optimal solution that is computed off-line, would render on-line optimisation unnecessary.
Recently, two approaches have simultaneously been developed, aiming at obtaining such off-line explicit solutions. These two approaches have been reported in, for example, [2] and [11]. The first method provides an algorithm, based on multi-parametric quadratic programming techniques, to obtain an explicit solution to the problem. The second method mentioned above uses geometric arguments to obtain a characterisation of the optimal solution of the resulting quadratic programme. Subsequently, many interesting extensions have been reported. For example, in [9] an explicit solution to a Min–Max MPC problem with bounded uncertainties is obtained; a suboptimal formulation that reduces the complexity of the solution is proposed in [7]; in [6] the infinite-time solution is computed by combining multi-parametric quadratic programming with reachability analysis. In fact, there exists a growing number of publications on this topic that reflects the interest that these problems have generated.
Starting from a different perspective to the ones mentioned above, a solution to the input-constrained case has been reported in [4]. This solution, obtained by using dynamic programming arguments, was of a local nature; i.e., valid in a region of the state space, and consisted in simply clipping the optimal unconstrained solution; i.e., u=-satΔ(Kx)u=-satΔ(Kx), where satΔ(·)satΔ(·) is the saturation function with bounds ±Δ±Δ. (Related work that utilises a different approach based on KKT optimality conditions has also been published in [8]). In [3], the region where this solution is valid was further characterised by a set of linear inequalities and it was shown that, inside this region, the controller u=-satΔ(Kx)u=-satΔ(Kx) effectively reaches the constraints, thus providing a nontrivial characterisation of the optimal solution. These results were used in [5] to obtain improved terminal constraint sets that guarantee closed-loop stability of model predictive control schemes.
In the present paper, the solution by dynamic programming is further extended to provide the global solution (i.e., valid in the entire state space) to the problem for arbitrary horizon N . The global solution is, of course, not just u=-satΔ(Kx)u=-satΔ(Kx) and can be concisely summarised by the expression View the MathML sourceu=-satΔ(L^N,ix+h^N,i) for x∈Xix∈Xi, where the set Xi⊂RnXi⊂Rn represents a region of a state-space partition Rn=⋃jXjRn=⋃jXj. The solution presented in this paper exploits the dynamic nature of the optimisation problem to obtain an analytical characterisation of the optimal control law. Although the final result (when evaluated at any given particular problem) obviously coincides, by optimality, with those obtained by other methods, the main contribution of the solution obtained by dynamic programming lies in that all the derivations required are analytical and, hence, the solution is given by closed-form expressions (i.e., all the expressions are closed-form functions of the data of the problem). One of the motivations of this approach is that it provides an alternative methodology, based on analytical derivations, to obtain the solution. It is envisaged that this alternative methodology could be amenable to being extended to related open problems, such as constrained control of non-linear systems, closed-form solution of the constrained continuous-time LQR problem, etc.
The remainder of the paper is organised as follows. In Section 2, the input-constrained LQR problem is formulated. In Section 3 the solution is provided, which comprises the control law structure and the regions of the state space where each component of the control law is valid. The derivation of the solution is done by dynamic programming and is included in the Appendix at the end of the paper. The solution is illustrated with an example in Section 4. Finally, Section 5 presents the conclusions.
The solution of the input-constrained LQR problem has been derived analytically by using dynamic programming techniques. By exploiting the dynamic nature of the optimal control problem, insights into the intrinsic structure of the optimal solution have been obtained which complement recent results that address this problem. An example has been included, which illustrates the structure of the solution.
