This contribution proposes a modified version of the Iterative Dynamic Programming (IDP) method. Two main differences to the original method are introduced. The new algorithm deals with discrete-time input–output models compared to continuous-time state–space models described by a set of ODE/DAE used in the original method. The main purpose of these modifications is to reduce computational load of the original method, estimate the process models more easily, and to enable its use on-line in receding horizon
Iterative dynamic programming (IDP) is a method of dynamic optimisation developed by Luus a decade ago (Luus, 1989 and Luus & Rosen, 1991). It has attracted the attention of many researches due to its many very favourable properties: is easy to implement, is quite robust, does not involve solution of a non-linear programming (NLP) even in the case of input constraints, is reported to be capable of finding a global optimum, and does not require any differentiation of process equations that is sometimes very difficult.
The main drawback of the method is the problem of dimensionality that results in enormous computational load. This is the reason why it is used mainly for determination of open-loop optimal control policies (Dadebo & Mcauley, 1995, Mekarapiruk & Luus, 1997 and Fikar, Latifi, Fournier & Creff, 1998) and so it is confined only to theoretical studies.
There are some similar methods that search through the control space in a manner similar to IDP and that are significantly faster (Carrasco & Banga, 1997). This is usually achieved with the compromise of obtaining cost values that are slightly worse than optimal. However, it is well known that the optimum usually lies in a very flat valley and that suboptimal control trajectories may be very different from the optimal one.
As only open-loop trajectories are determined, the method in its original form is unsuitable for on-line implementation. The presence of modeling errors and disturbances can quickly lead to changes in optimal control trajectory resulting in off-spec products. A solution is to apply the principles known from predictive control, namely receding horizon implementation and disturbance estimation.
In this article, we propose a modified IDP method that is suitable for on-line implementation. Unlike the original method, discrete-time input–output models are used. Discrete-time models are in general faster to simulate, which is the main obstacle in IDP. Also as IDP uses principles of control vector parametrisation and usually assumes piece-wise constant control trajectory, discretisation of states/outputs follows quite naturally. Input–output model descriptions are used to avoid problems with state estimation and observer design. Moreover, parameters of input–output models may be more easily identifiable on-line and thus may improve the method in adaptive fashion.
As process predictor, several discrete-time models may be used, ranging from simple linear to complex nonlinear models such as for example artificial neural networks (ANN). The use of ANN in combination with IDP has also been investigated by Tholudur and Ramirez (1995) where the network has been used to identify parameters of known non-linear characteristics of a process and subsequently to estimate continuous states of the process. The disadvantage of such an approach becomes noticeable with not all states measurable. Moreover, the original drawback of IDP, speed, remains.
The article is organised as follows. The next section gives the main results of this article — a modified IDP algorithm together with specification of models needed, and discussion about predictive framework implementation. The simulations in Section 3 deal with the simple multivariable model of a distillation column and with a biochemical reactor. In both cases, a different type of discrete-time model is used. Finally, discussion of the results and conclusions are presented in Section 4.
This contribution has dealt with a modification of the IDP method that can be used with discrete-time models and within receding horizon formulation. The process model has been described by discrete-time input–output models that can serve as multi-step predictors. The reason for this type of model is a dramatic decrease of computational load needed for a single IDP iteration and the subsequent possibility to use IDP on-line.
Because the method deals with input–output system formulation rather that with state-space, some parts of the original IDP have been modified. Also, the receding horizon formulation leads to different strategies for the termination of the method. The results have shown that on-line implementation of the modified method is possible. The replacement of the mathematical model by an equivalent discrete-time model in the optimisation step takes advantage of the high speed processing, since simulations involve only a few non-iterative algebraic calculations.
To show the properties of the method, two chemical engineering processes have been selected that are difficult to control by means of classical linear control methods. In both cases the computational time was about 1% of the original method.
The second example of distillation column control also showed some problems that may occur when controlling badly conditioned multivariable plants.
