از مدل کنترل پیش بین قوی تا کنترل بهینه تصادفی و برنامه ریزی پویا تقریبی: چشم انداز حاصل از یک سفر شخصی

Developments in robust model predictive control are reviewed from a perspective gained through a personal involvement in the research area during the past two decades. Various min–max MPC formulations are discussed in the setting of optimizing the “worst-case” performance in closed loop. One of the insights gained is that the conventional open-loop formulation of MPC is fundamentally flawed to address optimal control of systems with uncertain parameters, though it can be tailored to give conservative solutions with robust stability guarantees for special classes of problems. Dynamic programming (DP) may be the only general framework for obtaining closed-loop optimal control solutions for such systems. Due to the “curse of dimensionality (COD),” however, exact solution of DP is seldom possible. Approximate dynamic programming (ADP), which attempts to overcome the COD, is discussed with potential extensions and future challenges.

This paper reviews efforts in combining robust control with model predictive control. It is not meant to be a comprehensive survey but rather a review of personal sort that describes my own involvement in the research area and perspectives gained from it. It is thought to be appropriate for the current special issue, which is being put together in behalf of Prof. Manfred Morari, because the journey began at Caltech where I did my Ph.D. research under his supervision. It is not overboard to say that my research interest and style were shaped and molded largely during those times. Caltech during the mid and late 1980s was a vibrant place for those engaged in robust control research. The students who were fortunate to be there during that period enjoyed free access to some of the most recognized authorities on the topic. New theories and tools like the H∞ control and the structured singular value (SSV) μ were taught and discussed as they were being developed. My home department was Chemical Engineering, so the group may not have participated in pioneering the avant garde theories but enjoyed the privilege of having a firsthand chance to learn and apply them to chemical systems. Though the topic of robust control was dominating the Caltech's research activities, Prof. Morari's group was certainly aware of an important new development within the process control community called model predictive control (MPC). MPC was making a big splash among the industrial process control leaders like those at Shell Development, owing largely to its generality and ability to handle constraints, and was quickly becoming a hot topic among the academic researchers in the community. Given the group's deep engagement and expertise in robust control, it was natural for us to seek ways to incorporate the various concepts of robust control into MPC, in order to impart better robustness behavior to the MPC controllers. It was believed that the lack of explicit consideration by MPC was a major barrier to better and wider use of MPC by industry. This was not straightforward as it seemed because MPC, at least in its basic form embraced by the industrialists, is inherently a time-domain technique, whereas most of the robust control theories, e.g., SSV and loop shaping, had been developed in the frequency domain. Some of the analysis efforts therefore were limited to unconstrained linear MPC, which could be translated into a linear time-invariant (LTI) control law and therefore yielded to frequency-domain analysis tools. In addition, an important milestone work that came out of these efforts was the min–max MPC formulation ( Campo & Morari, 1987), which later would serve as a cornerstone for a large volume of research activities that followed in the next two decades. By the time I left Caltech in 1991 to start my own academic career, I had developed a real interest in the problem of designing a model predictive controller for systems with uncertain parameters, e.g., those bounded within a polytope. Though my thesis research at Caltech started in the robust control area (applying the SSV theory to the sensor selection problem), my research had moved significantly towards MPC by the time of graduation (formulating MPC in state space and coupling it with a state estimator). This interest has led to a journey that lasted more than 20 years, almost the entire period of the author's academic career up to now. This paper describes the journey and attempts to provide some perspective on the problem, including its importance, difficulty, current status, and future challenges. It is not easy to admit after working on it for two decades that the problem remains largely unsolved but it is the case for this problem. Along the way, a lot of insights have been gained and some partial solutions have come along. In fact, it was soon realized that this problem connects to the more general problem of stochastic optimal control and Markov Decision Process (MDP). It was also noticed that MPC may be inherently flawed to address the general class of the problem due to its open-loop optimal control formulation. Dynamic programming (DP) may be the only general method for it but it has its own problem known as the “curse of dimensionality (COD).” Therefore my works in this area for the second decade have attempted to connect the problems of robust MPC and stochastic optimal control with a new class of theories and techniques collectively known as approximate dynamic programming (ADP) ([Bertsekas, 2012] and [Powell, 2011]). Research efforts in ADP have mostly been driven by the computer science community and therefore translations and refinements as well as testing of these techniques became a central part of the my research efforts. The remainder of the paper is organized as follows. In Section 2, a brief historical perspective into the related topics including robust MPC and ADP will be provided. In Section 3, a representative problem will be defined so that the various methods can be discussed in a more technical manner. Model form, uncertainty dynamics, and objective function will be mathematically defined and applicable methodologies will be categorized. In Section 4, several different robust MPC formulations will be given and their strengths and limitations will be discussed. In Section 5, the more general approach of DP and ADP will be brought in. We will focus on key concepts rather than specific methodologies in order to provide a sense for the motivation and the current state of the ADP development. Section 6 concludes the paper with some final perspectives. It should be apparent from reading the introduction that this paper carries somewhat of a personal tone, both in its contents and style. Though an excuse can be made for such a choice in a special issue paying a personal tribute, an apology is asked for nevertheless.

This paper reviewed the development in finding a robust control formulation within the methodology of model predictive control. An effort started in the 1980s has led to many research topics including min–max predictive control and approximate dynamic programming. Though a complete solution is not there, significant insights and several very good suboptimal methods for a limited class of systems have been gained. One of the insights is that the conventional open-loop formulation of MPC is fundamentally flawed to address systems with uncertain parameters, though it can be made to give fairly good solutions with robust stability guarantees for special classes of problems. Dynamic programming may be the only general approach for obtaining optimal control solutions for systems with uncertain parameters. Due to the COD, however, the DPs cannot be solved exactly but only approximately. Deriving error bounds and performance guarantees for the approximate solutions may be important in this regard. The general problem of optimally controlling uncertain systems remains largely unsolved at this point. This applies to both the set-based deterministic worst-case formulation and the stochastic optimal-control formulation. The problem encompasses so many challenging aspects. In the estimation side, one must find a way to propagate the feasible parameter set or the conditional probability density function in the combined state/parameter space, given the incoming measurements. On the control side, optimal control calculations must account for how the propagation of the parameter set or its probability density affects the performance index. Most of all, the two aspects are intimately coupled and cannot be separated in developing a solution for the problem. Approximate dynamic programming (ADP) approach provides some practical ways to solve the DP and construct a near-optimal control policy. Many of the tricks and heuristics have been proposed along with some fundamental results but a complete theory for it is absent at this point and there may never be one. On the other hand, an incomplete theory with a large bag of tricks and experiences may still make it a viable approach. ADP is also a tool for developing “self-optimizing” simulation models that are needed in analyzing and making policy decisions for various supply chain and social dynamical systems involving agents with their own optimizing decisions.