This award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5).
This project develops new methods for analysis and optimization of Markov Decision Processes with general state and action sets. In the case of continuous time Markov Decision Processes, this project also studies problems with unbounded transition rates. A Markov Decision Processes is a fundamental stochastic optimization model broadly used in various applications including control and management of production and service systems. The methods to be developed in this project stand to fill important gaps left in the literature that are becoming increasingly more crucial to applications. These gaps are how to control natural and man-made systems with large state spaces and with the possibility of rapid changes of the system states.
This project focuses on three tasks. Task 1 will cover Markov Decision Processes with general state spaces, weakly continuous transition probabilities, and general action sets. This task will develop new theoretical concepts and methods. Moreover, since weakly continuous transition probabilities covers most inventory control and revenue management problems, this provides academics and practitioners with a general tool to solve such problems. Task 2 will study continuous time problems with unbounded transition rates. Even in the case when the state space is discrete, this general problem has been open for quite some time. However, in the areas of parallel processing and distributed computing it is becoming increasingly more common to not have an a priori bound on the action set. Our work will address these issues. The third task is dedicated to applying our results from the first two tasks to several areas. We shall investigate stochastic inventory control and revenue management problems. If successful, this project will develop optimization methods for broad classes of stochastic systems and provide theoretical and computation foundations for solutions of large-scale inventory and revenue management problems.
