The research objective of this award is to develop computationally efficient algorithms for solving certain classes of Markov Decision Processes (MDPs). MDPs, also known under the name of stochastic dynamic programming, provide major operations research methods and tools for dynamic optimization of stochastic systems. They are broadly used for the optimization of production, service, telecommunication, and military systems. In addition to operations research, MDPs are used in many other disciplines including electrical engineering, computer science, and economics. The project will investigate the problems with the two most often used objective criteria: average costs per unit of time and the expected total discounted costs as well as risk sensitive criteria. In addition to problems with a single objective criterion, it will investigate problems with multiple criteria and constraints. The computational efficiency will be studied with respect to the parameters of an MDP and with respect to the parameters of the initial problem formulation.
If successful, the results of this research will provide new methodologies and computer algorithms for finding exact and approximate solutions to MDPs modeling applications to production and service systems. These applications include inventory and queueing control, scheduling, and resource allocation. This project will also contribute to the development of human resources in science and engineering. First, it will support Ph.D. students at the Stony Brook University to conduct research related to this project. Second, it will create research and educational projects for graduate and undergraduate students including minority and female students in science and engineering. The results of this project will be used in applied probability and dynamic programming courses that the PI teaches. The results of this project will be disseminated via journal publications, the internet, and included into the text "Introduction to Markov Decision Processes" the PI is currently working on.
