برنامه ریزی مستمر و پویا Lipschitz با تخفیف

We show that if the return function, the technological constraints and the transition function of a standard problem of stochastic dynamic programming with discount satisfy Lipschitz regularity assumptions, then the value function is Lipschitz regular.

The results of this paper stand for a class of dynamic optimization problems with infinite horizon and discount, in a stochastic setting, as described by Stokey et al. [29, Chapters 4, 9]. It is well known that, under topological assumptions (compactness and continuity) on the data of the problem (i.e., state space, return function and technological constraint correspondence), the existence, uniqueness and continuity of the value function is guaranteed. The theory of dynamic programming with discount proceeds by completing the topological assumptions with a rather extensive block of assumptions, which we call standard assumptions, including concavity, smoothness and monotonicity of the data. Such assumptions guarantee the concavity, smoothness and numerical computability of the value function and optimal policy correspondence. In the non-random case they also guarantee the convergence of the optimal paths to an equilibrium state, and if non-interior optimal paths are ruled out, then a recursive computation of the optimal paths through Euler equations is possible. The examples in Section 4 show that all these nice properties, with exception of the existence and continuity of the value function, fail to hold under small departures from the standard assumptions. A variety of phenomena emerge there related with non-concavity of the objective function, as countably many points of discontinuity and non-uniqueness, following a systematic pattern, in the optimal policy correspondence; discontinuities in the form of jumps upwards in the marginal value function, synchronized with the discontinuities of the optimal policy correspondence (Example 18); asymptotic cyclic behavior of the optimal paths (Example 18). In these cases, not only do the properties derived from the standard assumptions are hopelessly lost, but also the numerical computation of the value function through Bellman operator iterates is not possible for states out of the grid used for the discretization of the phase space, since no rate of convergence can be derived from the standard theory for such states. Is it possible to construct an alternative theoretical framework which does not require the extensive list of standard assumptions? Can such theory give useful information on some relevant problems that are intractable in the standard framework? The aim of this paper is to give some partial answers to these questions. In particular we show that if the data of the problem satisfy Lipschitz continuous assumptions, then the value function is Lipschitz continuous (see Theorem 14). A first consequence of our result is that in that setting the value function and optimal policy correspondence are numerically computable (see [26]), thus giving a theoretical basis to our examples above, and to some numerical experiments that have recently raised interest in the literature [11] and [12]. The most direct antecedent of this paper is the result of Bertsekas [3]. There, in a setting of optimal stochastic control with a discrete state space for the random shocks and admissible controls, it is proved that under Lipschitz assumptions on the data of the problem, the value function can be computed. If we ignore the different settings of the problems, the main contribution of our results is that Bertsekas’ method does not permit us to prove that the value function is Lipschitz regular, which is the key step to the obtention of the rate of convergence of the numerical algorithm for the computation of the value function [26]. The Lipschitz continuity of the value function has been analyzed by Yue [30] and [31] in optimal control and optimal time control problems respectively. Montrucchio [25] proves that the policy function is Lipschitz continuous under assumptions of strong concavity. Bardi and Capuzzo-Dolceta [2] proved that the value function is Lipschitz continuous in infinite horizon problems of optimal control with discount. This result does not allow dependence of the admissible controls on the state of the system, i.e., the existence of a technological constraint correspondence is ruled out. Such correspondence plays a central role in the problem. Relevant research regarding applications of the results in this paper is focused on non-concavity of the growth function of the resource in problems of optimal exploitation of renewable resources [8]. Note that these problems are equivalent to optimal growth models with linear or strictly concave objective functions and convex–concave production functions [22], [23], [13] and [21]. In this sense, the results in this paper can also be applied to these economic problems. A second field where the results of this paper find natural application is that of dynamic optimization problems with convex objective functions. This case has been shown to be relevant in the exploitation of renewable resources. In particular, there is empirical evidence in the literature about convex objective functions in fisheries management [5] and [6]. See also Dasgupta and Mäler [10] for a recent overview on non-convex ecosystems, and Dawid and Kopel [11] and [12] for a numerical analysis of a renewable resource subject to a convex objective function. The case of a convex objective function is also relevant in capital accumulation models of the firm where the revenue is a convex function of the capital stock [15]. See also Hartl and Kort [16] for capital accumulation models where the revenue is a convex–concave function of the capital stock. Lastly, there are relevant economic problems that may be treated in the Lipschitz setting. See Arrow et al. [1], Brian [7], and Heal [17] for recent overviews on economic problems related to non-concavities. See also references in Example 18.

There are three directions for the future extension of this research: (1) In Morán and Maroto [26], we analyze the convergence of the algorithm for the numerical computation of the value function. The Lipschitz continuity of the value function is the only assumption that we need in order to obtain a rate O(δ)O(δ) of convergence of the numerical algorithm, where δδ is the diameter of the discrete grid of points used for the computation. (2) The results in this paper for the case of interior optimal plans allow us to prove the Lipschitz continuity of the value function for the case of eventually interior optimal plans. This case is relevant to the area of optimal exploitation of renewable resources where the existence of non-concavities has been largely admitted. (3) As a future perspective, the results in this paper allow us the use of powerful tools of non-smooth analysis [9]. In the same spirit, an incipient development of a theory of dynamical systems with evolution law governed by correspondences [19] and [20], closely related to fractal geometry [18], open the perspective of the analysis of the long run behavior of optimal policies and asymptotic stability of dynamical equilibria if non-uniqueness in the policy correspondence is allowed, so in the deterministic as in the stochastic settings. For recent research on chaotic behavior of optimal paths, see also Majumdar et al. [24].