برنامهریزی پویا بر اساس بهینه سازی تولید انورتاز در یک راکتور دسته تغذیه با استفاده از شبیه سازی همراه با طبقه بندی های عصبی

A controller based on neuro-dynamic programming coupled with a fuzzy ARTMAP neural network for a fed-batch bioreactor was developed to produce cloned invertase in Saccharomyces cerevisiae yeast in a fed-batch bioreactor. The objective was to find the optimal glucose feed rate profile needed to achieve the highest fermentation profit in this reactive system where the enzyme expression is repressed at high glucose concentrations. The controller updated in time an optimal control action that incremented the fed-batch bioreactor profitability. The proposed neuro-dynamic programming (NDP) approach, coupled with fuzzy ARTMAP classifier, utilized suboptimal control policies to start the optimization. The fuzzy ARTMAP algorithm was used to build a cost surface in the state space visited by the process, thus minimizing the curse of dimensionality with the associated high computational costs. Bellman's iteration was used to improve the fuzzy ARTMAP approximation of the cost surface before its implementation into the control system. The controller was tested at different fermentation conditions for initial reactor volumes within the range 0.4–0.8 l and a final constant fermentation volume of 1.2 l. Profits were higher than those previously reported in the literature, with continuous and smooth glucose feed rate profiles easy to implement under production conditions. The control system was also tested when the substract concentration changed unexpectedly. The controller global performance was also in this case better than those obtained with the best suboptimal policy and previous methods.

Many industrial fermentation processes involving production of antibiotics, enzymes and organic acids are carried out in a fed-batch mode of operation, where substrates are added continuously. Fed-batch bioreactors are particular useful when the growth and/or metabolite production is inhibited at certain substrate or end-product concentrations or due to a catabolite repression. In those cases, the controlled addition of substrate is essential to achieve maximum production of the desired product, i.e., it is necessary to determine the optimal substrate feed rate profile (Balsa-Canto, Banga, Alonso, & Vassiliadis, 2000; Georgieva, Hristozov, Pencheva, Tzonkov, & Hitzmann, 2003; Riascos & Pinto, 2004; Smets, Claes, November, Bastin, & van Impe, 2004; Stigter & Keesman, 2004; Zhang & Lennox, 2004). The problem of determining the optimal substrate feed rate profile is a singular control problem. The control variable, substrate feed rate, usually appears linearly coupled with the state equations that describe the process. Many optimization methods commonly used to solve the singular control problem do not work well in systems described by more than four differential equations. This is the case of the fermentation process for the cloned invertase expression in Saccharomyces cerevisiae yeast studied by Patkar and Seo (1992). These authors found that the enzyme expression was repressed when the substrate concentration was high. They also investigated the fed-batch operation of the bioreactor with the aim of increasing productivity. Patkar and Seo (1993) proposed later a bioreactor model that takes into account the respiratory and the fermentative fluxes for the substrate consumption. With the help of this model they used the conjugate gradient method to find the optimal feed rate profile for certain fermentation process conditions. Chaudhuri and Modak (1998) incorporated a neural network model into the generalized reduced gradient method for the same productivity optimization. Another alternative is to use genetic algorithms for the optimization problem ( Sarkar and Modak, 2004 and Sarkar and Modak, 2005). The optimization methods applied previously by Patkar and Seo (1993) and Chaudhuri and Modak (1998) require the solution of a new and different optimization problem for each initial condition because the fermentation ending time has to be fixed before the optimization procedure is carried out. Thus, several fermentation ending values have to be tried to find the optimal fermentation time, and for each one of them the productivity has to be optimized; this involves many operations and is computationally demanding. In addition, if the fermentation process changes its state due to unknown disturbances, these fixed feed rate policies used previously cannot drive the system back towards an optimal final productivity value. Neuro-dynamic programming (NDP) is an optimization method that can be used to determine the optimal final fermentation time in fed-batch bioreactors as part of the optimization process when an objective function accounting for the ending time is adopted (Valencia, Kaisare, & Lee, 2005). Dynamic programming (Bellman, 1957) is an approach to model dynamic decision problems, to analyze the structural properties of these problems, and to solve them. The fermentation process under study is envisaged and modeled as a chain of consecutive transitions from one state to another. The modeled process always occupies a state at each point in time, i.e., it can be viewed as infinite or finite time horizon problem depending on the amount of time steps considered. The way each transition is completed depends on the control or decision variable and, in stochastic problems, on a transition probability function. Each effected action or decision made has an associated cost or reward. The objective of dynamic programming is to minimize the total incurred cost, obtained from the sum (or product) of the cost of the transitions needed to reach the final desired process state from an initial process state. The set of all the decisions made is called a policy. An optimal cost is obtained through a series of optimal actions. Thus, an optimal cost has an associated optimal policy. The applicability of dynamic programming (Bellman & Dreyfus, 1962) to many important practical problems is limited by the enormous size of the underlying state space. This limitation was first pointed out by Bellman and it is know as the Bellman's curse of dimensionality. Neural networks have been used to infer the state space from examples (Bertsekas & Tsitsiklis, 1996; Desai, Badhe, Tambe, & Kulkarni, 2006; Valencia et al., 2005) and overcome this limitation. The coupled utilization of neural networks with dynamic programming is called neuro-dynamic programming or reinforcement learning, which is the term used in the artificial intelligence literature. In the current study the dynamic programming approach is used in conjunction with a fuzzy ARTMAP neural system to solve the singular control problem of a fed-batch fermentation process for cloned invertase production in S. cerevisiae yeast. The aim is to find the optimal control action at any time during the fermentation process, i.e., finding the maximum productivity with the minimum total fermentation time for different initial bioreactor volumes. Section 2 introduces the NDP methodology, the fuzzy ARTMAP architecture and the optimal control model. The methodology followed to optimize the production of invertase with the current model is explained in detail in Section 3. Finally, the optimal controller performance and conclusions are presented in Sections 4 and 5, respectively.

The study of the controlled manipulation of the glucose feed rate into a fed-batch bioreactor to produce cloned invertase in S. cerevisiae yeast has shown that neuro-dynamic programming coupled with fuzzy ARTMAP classifier is a good alternative to standard optimization methods previously applied to fermentation processes. The fuzzy ARTMAP-NDP controller, which starts all optimizations from suboptimal feed rate control policies and results in incremental smooth changes in the feed rate profiles, yields profits that outperform those reported previously in the literature. The interpolation of the cost surface in the state space performed by the fuzzy ARTMAP algorithm together with the Bellman's iteration approach minimizes the curse of dimensionality of the calculations and facilitates the development of a control system that is robust to disturbances like the change of substrate concentration in the bioreactor at a given time. The feeding profiles obtained are similar in trend to the ones reported previously in the literature and also implemented experimentally. The proposed fuzzy ARTMAP-NDP methodology could be readily applied to a wide range of processes by selecting objective functions that could best capture the non-linear nature of each control problem.