We study the ergodic control problem related to stochastic production planning in a single product manufacturing system with production constraints. The existence of a solution to the corresponding Hamilton-Jacobi-Bellman equation and its properties are shown. Furthermore, the optimal control for the ergodic control problem and an example are given.
This paper deals with the following first-order nonlinear differential equation:
-i~(x,i)+Aw(x,i)+h(x), x E R1, i=1,2 )...) d. (1)
Here, X is a constant, F(x) = kx if x < 0, = 0 if x 2 0 for some positive constant k > 0, h
is a convex function, and A denotes the infinitesimal generator of an irreducible Markov chain
(z(t), P) with state space 2 = {1,2,. . . , d},
A+, i) = c q&(x,j) - v(x, i)],
j#i
(2)
where qij is the jump rate of z(t) from state i to state j. The unknown is the pair (II, A), where
w(.,i) E C’(R1) for every i E 2.
Equation (1) arises in the ergodic control problem of stochastic production planning in a
single product manufacturing system and is called the Hamilton-Jacobi-Bellman (HJB) equation
or dynamic programming equation. The inventory level s(t) of stochastic production planning
modelled by Sethi and Zhang [l, Section 3.5, p. 50) is governed by the differential equation
dx(t)
- = p(t) - da dt
x(0) = 2, t(0) = i, P-a.s.,for production rate 0 I p(t) 5 k, in which z(t) and k are interpreted as the demand rate and
capacity level, respectively. For the ergodic control problem, the cost J@(e) : rc,i) associated
with p(e) is given by
J(p(.) : z, i) = lim sup LE
T-co T
l+(t)) dt 1 z(O) = 2, z(0) = i 1 ,
where h(z) represents the convex inventory cost.
The purpose of this paper is to show the existence of a solution to the HJB equation (1) and
to present the optimal control minimizing the cost J@(e) : z, i) subject to (3). In the control
problem of stochastic production planning with discounted rate a > 0, Bensoussan et al. [2],
Fleming et al. [3], and Sethi et al. [4] have investigated the HJB equation
For the ergodic control problem, we study the limit of (5) as a tends to 0, and investigate a solution
to the degenerate HJB equation (1). This approach develops the technique of Bensoussan and
Frehse [5] concerning nondegenerate second-order partial differential equations to our degenerate
case.
Section 2 is devoted to the existence problem of the HJB equation (1) under the convexity
assumption and others on h, and properties of the solution are shown in Section 3. In Section 4,
an optimal control for the ergodic control problem and the value are given. In Section 5, we
present an example of the solution to the HJB equation (l), and the optimal control and the
value are given.
In this paper, we have examined the ergodic control problem for the stochastic production
planning with production constraints. We show that there exists a solution (v, A) to the corre-
sponding degenerate HJB equation under some conditions, and properties of the solution (v, A)
are investigated. Moreover, the optimal control p* for the ergodic control problem is given in
term of the partial derivative $$ of the solution (w,X) to the HJB equation, and the value is
given by constant X of the solution (u, A). Finally, more work needs to show the uniqueness of
the solution to the HJB equation.
