9625489 Tsitsiklis This project deals with problems of sequential decision making under uncertainty that are too large for the classical methods of dynamic programming. The focus is on approximate dynamic programming which is a rich field that combines classical dynamic programming, reinforcement learning and other methods from artificial intelligence, approximation theory and neural networks, and simulation. The objectives of the research are as follows: (1). Resolve several outstanding question related to simulation-based methods involving lookup table representations. (2). Provide a theoretical understanding of approximate dynamic programming methods, when a compact representation of the cost-to-go function is employed. In particular, identifying classes of algorithm/architecture combinations for which convergence is guaranteed, together with error bounds. (3). Develop the theory and algorithms for average cost approximate dynamic programming. (4). Develop the theory and algorithms for approximate dynamic programming for the case of Markov games. (5). Carry out a number of case studies that will provide a better understanding of the comparative performance of different methods, and to demonstrate the usefulness of the methods on realistic problems of real-world importance. Three case studies involve: (a) Dynamic scheduling of a fleet of trucks (b) Pricing of complex derivative financial instruments. (c) The game of football. The overall objectives are to provide a comprehensive mathematical foundation for the field of approximate dynamic programming and at the same time succeed in solving some difficult and important problems.
