کنترل مبتنی بر برنامه ریزی تقریبی دینامیکی سیستم های هذلولی PDE با استفاده از مدل های سفارش کاهش یافته از روش مشخصه ها

Approximate dynamic programming (ADP) is a model based control technique suitable for nonlinear systems. Application of ADP to distributed parameter systems (DPS) which are described by partial differential equations is a computationally intensive task. This problem is addressed in literature by the use of reduced order models which capture the essential dynamics of the system. Order reduction of DPS described by hyperbolic PDEs is a difficult task as such systems exhibit modes of nearly equal energy. The focus of this contribution is ADP based control of systems described by hyperbolic PDEs using reduced order models. Method of characteristics (MOC) is used to obtain reduced order models. This reduced order model is then used in ADP based control for solving the set-point tracking problem. Two case studies involving single and double characteristics are studied. Open loop simulations demonstrate the effectiveness of MOC in reducing the order and the closed loop simulations with ADP based controller indicate the advantage of using these reduced order models.

Several typical chemical engineering systems such as fixed bed reactors, plug flow reactors, etc. are inherently nonlinear. The dynamic behavior and spatial variation of key state variables are accurately captured by first principle partial differential equation (PDE) models. These PDE models are valid in a larger operating range as compared to lumped parameter ODE models and have been extensively used for design and analysis. It is expected that use of such PDE models in model based controllers will result in improved closed loop performance. However, the use of these models in online control applications is limited. There are two key issues that need to be addressed: the computational effort involved in solving nonlinear optimal control problems and the fact that the state space form of the PDE models is infinite dimensional. Typical model based controllers such as model predictive control (MPC) require solution of a multi-step optimization problem with a finite horizon cost. An alternative is to formulate a dynamic programming problem which involves the solution of single step optimization problem with an infinite horizon cost. DP employs the ‘Bellman principle of optimality’ to obtain the ‘optimal cost-to-go’ values which is the optimal cost involved to reach the set-point starting from current state. This cost-to-go value is obtained for all the points in the discretized state space. Function approximators are used to get the map between cost-to-go values and the points in the state space. The exponential increase in the number of points in state-space as the state dimension increases which is called ‘curse of dimensionality’ in literature is addressed by formulating an approximate version, viz., ADP by focusing on the relevant state space region traced by other sub-optimal controllers. The cost-to-go values approximate the optimal infinite horizon cost and hence the use of longer horizon can be avoided, thereby reducing the computational load significantly. In recent years, ADP has been successfully applied to several applications such as a complex bio-reactor characterized by multiple steady states (Kaisare, Lee, & Lee, 2003), integrated plants involving a reactor and distillation column with a recycle (Tosukhowong & Lee, 2009), systems described by partial differential equations (Midhun and Kaisare, 2011 and Padhi and Balakrishnan, 2003), etc. However application of ADP to distributed parameters system is a challenging task because of the high computational effort in solving PDEs. In addition, a high order state space representation of DPS results in computational issues in obtaining cost-to-go estimate using function approximators (Lee, Kaisare, & Lee, 2006). This necessitates the use of reduced order models for application of ADP based control to DPS. Model order reduction techniques generate a finite dimensional state space model that describes the original system dynamics fairly accurately while requiring lower computational effort. There are several order reduction techniques that have been proposed to develop reduced order models suitable for control. Method of lines (MOL) is a conventional order reduction method where the spatial derivative at each nodal point is calculated through appropriate finite difference based approximation (Dochain et al., 1992, Midhun and Kaisare, 2011 and Sorensen et al., 1980). Proper orthogonal decomposition (POD) is another popular method which is suitable for parabolic PDE systems (PDEs involving a second order spatial differential operator) which exhibit modes of different energy. Reduced order models generated using POD have significantly lower dimension compared to those generated by the finite difference method and hence been extensively used in the design of finite dimensional controllers for systems described by parabolic differential equations (Padhi and Balakrishnan, 2003, Pitchaiah and Armaou, 2010, Shvartsman et al., 2000 and Shvartsman and Kevrekidis, 1998). Convection dominated systems such as plug flow reactors, fixed bed reactors, heat exchangers, etc., are described by a set of first order hyperbolic PDEs. The use of finite difference method for such system requires very large number of states to satisfactorily describe the system. Also order reduction through modal decomposition is not possible as the spatial operator of hyperbolic PDE has modes with nearly equal energy (Christofides & Daoutidis, 1998). Hence optimal control techniques which retain the infinite dimensional nature of the systems have been reported (Balas, 1986, Choe and Chang, 1998, Lo, 1973 and Wang, 1966). However design and implementation of such infinite dimensional optimal control techniques is complicated. On the other hand, finite dimensional controllers are very well developed and hence there is a need to develop a method which reduces the order of the first order hyperbolic PDEs for design of finite dimensional controllers. Method of characteristics (MOC) is a conventional solution method for solving first order hyperbolic PDEs and has also been used in the model based control. A combination of MOC with sliding mode control was proposed for nonlinear hyperbolic systems (Hanczyc & Palazoglu, 1995). A output feedback control methodology for systems described by a first order hyperbolic PDE has been developed (Christofides & Daoutidis, 1996). Lyapunov based robust controller was developed for nonlinear hyperbolic PDE (Christofides & Daoutidis, 1998). A combination of finite difference method with MOC has been employed for the control of convection dominated parabolic systems (Shang, Forbes, & Guay, 2007). Finite difference schemes which retain the FIR property of co-current reactor have been proposed and used for the design of controllers (Choi, 2007, Choi and Lee, 2004 and Choi and Lee, 2005). Model predictive control (MPC) based on MOC for fixed bed reactor, plug flow reactor and catalytic flow reversal reactor have been proposed in literature (Fuxman et al., 2007, Mohammadi et al., 2010 and Shang et al., 2004). Though there are applications involving MOC, order reduction using MOC and its subsequent use in model based control is not discussed in the literature. However, the focus of these has been use of MOC as a solution technique, rather than a model order reduction technique. Further in these works, simple approximations such as assuming constant values for the nonlinear terms in the resulting ODEs is employed. In our work we propose an improved method which results in a accurate reduced order model of significantly lower order and complexity. The primary focus of this article is two-fold. One involves proposing better approximation in the implementation of MOC to hyperbolic PDEs which results in reduced order model. As already mentioned ADP has the advantage of improved closed loop performance with reduced computational load and applying this concept to PDEs systems requires reduced order models. So second part of this work involves applying the reduced order model from MOC in ADP based controller. In the closed loop simulation, we allow for plant-model mismatch by using reduced order model for prediction and optimization and higher order model from MOL as a ‘plant’. Though the use of reduced order from MOC in ADP requires interpolation at every sampling instant, we have shown that the resulting closed loop response shows better behavior compared to the nonlinear MPC. To summarize, the contribution of the paper is a new methodology to obtain reduced order models using MOC for systems described by quasi-linear hyperbolic PDEs. Further the robustness of this reduced order model is illustrated through the closed loop simulation involving ADP based control. The organization of this paper is as follows: Initially the theory behind approximate dynamic programming and the problem in using high dimensional state space models is explained. Next, method of characteristics as a order reduction method is introduced and the methodology of obtaining reduced order models through MOC is explained in detail. This is followed by presentation of the closed loop simulation results using ADP employing MOC for the two case studies involving single and double characteristics. The overall advantages and shortcomings in employing reduced order models from MOC in ADP based control for DPS system are then summarized.

The use of method of characteristics to obtain reduced order models for ADP based control of systems described by quasi-linear hyperbolic PDEs was demonstrated. The use of reduced order model reduces the computational complexity and increases the reliability of the function approximator used in ADP. Since hyperbolic PDE exhibits modes of nearly equal energy, methods that rely on decomposition of Eigen-modes are not applicable for model reduction. The use of MOC to obtain lower dimensional models is illustrated through two case studies, involving single and two characteristic lines, respectively. The density of characteristic curves determines the accuracy of the solution; along with the slope of the characteristic lines, it also determines the time instances where the solution variables are computed. A new methodology was proposed, which uses numerical integration for the solution along the characteristic lines, combined with linear interpolation to obtain the solution at the desired sampling times. Although the implementation gets more complicated as the number of characteristic curves increases, several engineering systems of interest can be modeled by PDEs involving one or two characteristics. The use of MOC-based reduced order model within ADP framework was demonstrated. It has been previously reported that capturing the initial transient response and the steady state gain accurately is needed for optimal performance of ADP-based control. In spite of the slight plant-model mismatch (due to lower density of characteristic curves and interpolation used), ADP showed better performance than the NMPC controllers. The proposed ADP-MOC method operated by finding an optimal path within the region of the state space traced by the initial suboptimal control laws. In summary, the ADP-based control of quasi-linear hyperbolic PDE can be improved by using MOC as a model order reduction method.