9404177 Yushkevich The Theory of Markov Decision Processes (MDP's) is an important tool in operations research, management sciences, and numerical solution of continuous time stochastic control problems. Sensitive criteria in MDP's permit adjustments for the underselectiveness of the average per unit time reward criterion in cases where initial stages are important and they also provide deeper insight into the nature of optimality conditions and equations. Up to now, sensitive criteria have been studied only for MDP's with finite or countable state spaces. Models with a continuous state space are more relevant in many applications, for instance, when there are incomplete observations. The purpose of the proposed research is to extend the theory of sensitive criteria to MDP's with a Borel state space. First of all models with an absolutely continuous transition function will be studied. For such models, we expect to obtain (by means of limit theorems for general state space Markov chains, Laurent expansions of resolvents, and, especially, new techniques for aggregation and compactification of the set of feedback controls) the following results: 1)validity of the lexicographical optimality equation, 2) existence of deterministic stationary sensitively optimal policies, and 3) effectiveness of a lexicographical policy improvement algorithm to get such a policy. Markov Decision Processes provide important tools for the analysis of many control and management optimization problems in which randomness plays a significant role. They are often used to optimize an operation's resource allocations among competing requirements (for example, inventory costs in maintaining a certain number of spare parts for a system versus down time costs for the system versus costs of repairing defective parts of the system in a given amount of time). In many of these problems, the long-run average cost (or reward) is the natural objective function to optimize. However, long-run average criteria are in sensitive to costs or rewards associated with initial stages, which, under certain circumstances, can lead to nonoptimal policies. Sensitive criteria have only been successfully analyzed for the case when the possible states of the system are finite or countable. There are many realistic situations, however, when the appropriate state space is a continuum, for example, when incomplete observations are involved, which is often the case. It is the purpose of this research to extend the theory of sensitive criteria to Markov Decision Processes with Borel state spaces (which are general enough to include all known situations of importance).
