برنامه ریزی مستمر دینامیکی لیپشیتز با تخفیف II

We construct an alternative theoretical framework for stochastic dynamic programming which allows us to replace concavity assumptions with more flexible Lipschitz continuous assumptions. This framework allows us to prove that the value function of stochastic dynamic programming problems with discount is Lipschitz continuous in the presence of nonconcavities in the data of the problem. Our method allows us to treat problems with noninterior optimal paths. We also describe a discretization algorithm for the numerical computation of the value function, and we obtain the rate of convergence of this algorithm.

In this paper we complete the treatment of problems of stochastic dynamic programming with discount in a framework of Lipschitz continuous hypothesis on the data of the problem. Dynamic programming with discount provides a setting for the analysis of optimal intertemporal transfers of economic resources. There is assumed the existence of a central planner who tries to maximize, over all feasible currents c1,c2,c3…c1,c2,c3… of future consumptions, View the MathML source∑i=1∞βiER(ci), where View the MathML sourceER(ci) is the expected return at period ii derived from consumption cici and β∈(0,1)β∈(0,1) is the discount factor (see Section 2 for a full exposition of the problem). Typically RR is a monetary benefit or some subjective utility which summarizes the central planner’s objective, and ββ reflects the willingness to substitute between present and future return. Some of the principal models in today’s macroeconomic theory as described by Ljungqvist and Sargent [13] are expressible in this framework. Also, many problems at the microeconomic level are currently treated in this setting (see [19]), in particular, problems of optimal exploitation of renewable resources (see Example 10). The standard theory of dynamic programming with discount relies heavily on the concavity of the data of the problem (i.e., state space, return function and technological constraint correspondence). It first requires compactness and continuity of the data in order to guarantee the existence, uniqueness and continuity of the value function. Concavity, smoothness and monotonicity are then required in order to guarantee the smoothness and numerical computability of the value function and the optimal policy correspondence. In the deterministic case, these assumptions also guarantee that for discount factors close to 1 the optimal paths converge to an equilibrium state (the so-called turnpike theory). Another standard assumption is requiring always interior optimal paths; this allows the recursive computation of the optimal policy through Euler equations. See Stokey et al. [19], Chapters 4, 9, for a detailed analysis of the standard theory. There are, however, economic and environmental problems that present nonconcavities. Empirical evidence regarding this is given below. See also Maroto and Moran [15] for a discussion of the literature on these problems. The standard assumptions, with the exceptions of compactness and continuity, fail in this setting. The value function can be nonconcave and nonsmooth, and even the numerical analysis lacks a theoretical basis, since no rate of convergence of the algorithms can be obtained from the standard theory. In Maroto and Moran [15], we construct an alternative theoretical framework based solely on the general hypothesis of Lipschitz continuity of the data. We compute useful information regarding these problems in the case of always interior optimal plans, a case that is relevant in problems of economic growth and in problems of exploitation of renewable resources in which a null or a total consumption is always suboptimal. There are, however, problems in which the optimal choices for early transitions may be noninterior, whereas the optimal selections are interior at some later transitions. We shall refer to such cases as eventually interior optimal plans. Examples of this situation are provided by problems of optimal exploitation of renewable resources in which, due to the presence of increasing marginal returns, it might well be optimal to let the resource grow freely for some periods and then to carry out a large harvesting. In these cases, the optimal policies might be in fact cyclical with periods of null harvesting (see Examples in Section 5). In this paper we extend the results in Maroto and Moran [15] to the case of eventually interior optimal plans. We establish conditions of Lipschitz continuity on the data (Section 2.2) of a standard discounted dynamic programming problem, in a stochastic setting. Our main result is that in such a setting the value function is Lipschitz continuous (Section 3). This establishes a basis for an analytical study of these problems through the tools of nonsmooth analysis. Our first application of the Lipschitz continuity of the value function of the problem is to show that the discretization algorithm for the computation of the value function derived from this theory converges with a rate O(δ)O(δ), with δδ being the maximum diameter of the simplices of the discretization net. We then test the robustness of our results via the application of the algorithm to the study of the optimal management of a renewable resource (Section 5). We show that nonconcavities in the data of the problem can lead to conclusions differing dramatically from those of the standard theory. In particular, cycles may exist in the optimal policy dynamics instead of there being a steady state equilibrium. Research to which the results in this paper can be applied includes studies of the optimal exploitation of schooling species, especially clupeids. Bjørndal and Conrad [2] estimated a harvest function for North Sea herring and they found increasing marginal returns (nonconcavities). Similar results were found earlier by Hannesson [10] for the North Atlantic cod fishery. See also references in Examples (Section 5). The schooling species gather in large banks (schools), a behavior which reduces the effectiveness of predators (Partridge [18]). Schooling behavior and the modern fish-finding technology incorporated in fishing vessels make efficient localization and harvesting of these species possible. This gives rise to a nonconcave net revenue function. See Dawid and Kopel [6] and [7] for the optimal exploitation of a renewable resource subject to a convex return function. A second field where the results of this paper find natural application is that of optimal exploitation of renewable resources with a nonconcave growth function that exhibits depensation (“S-shaped”). According to Clark [5], schooling behavior may give rise to such cases. See also Clark [4], Majumdar and Mitra [14], Dechert and Nishimura [8], Le Van and Dana [12], and Olson and Roy [17], for treatment of these problems.

In this paper we have completed an alternative theoretical framework which allows us to analyze problems of stochastic dynamic programming with discount in the presence of nonconcavities in the data of the problem. The mathematical complexity inherent in the proofs is compensated by the wide applicability of the results and by the amenability to numerical analysis of the algorithm derived from this theory. This algorithm allows us to analyze numerically problems of optimal exploitation of renewable resources in danger of collapse which are intractable in the standard framework. This algorithm also allow us to explore properties of the optimal policy correspondence related with nonconcavities of the data of the problem, such as countably many points of discontinuity and nonuniqueness following a systematic pattern; discontinuities in the form of jumps upwards in the marginal value function, synchronized with the discontinuities of the optimal policy correspondence; asymptotic cyclic behavior of the optimal paths; existence of a threshold level beyond which there are harvests; and existence of a threshold level below which the resource stock is in danger of collapse even without harvesting. Notice that each of these phenomena is an object of study itself that is potentially analyzable taking the results of this paper as a starting point, and using the available tools of nonsmooth analysis.