Many systems arising in applications from engineering design, manufacturing, and health care require the use of simulation optimization techniques to improve their performance. However, despite significant progress in recent years, simulation optimization remains an area with many theoretical and practical challenges. This research project aims to expand the current knowledge in this field by investigating a novel approach that integrates theories and tools from reinforcement learning (a subarea of artificial intelligence) within a class of adaptive search algorithms called the model-based methods to solve simulation optimization problems. Because of the generality of these methodologies, the resulting techniques will have broad applicability in a wide array of industry and science sectors. In particular, through collaboration with power engineers, the developed algorithms will be tested and applied to voltage control problems in electric power systems, potentially benefiting both utility companies and energy consumers. The research plan will be closely integrated with the education and training of students in engineering by incorporating new developments into the graduate courses the investigator teaches and recruiting female and underrepresented minority students to the project.
The goal of this research is to advance theoretical underpinnings of new model-based algorithms that can be orders of magnitude more efficient than the state-of-the-art. This will be accomplished by exploring the connections between model-based methods and policy gradient-based reinforcement-learning algorithms. Specifically, the investigator will examine how to use the insights from actor-critic algorithms in the reinforcement learning framework to effectively reduce the sampling variance of model-based methods. If successful, the approach will integrate function approximation techniques within a model-based optimization setting to provide algorithms with low-variance performance estimates in searching for improved solutions. This research may change the manner in which these algorithms are implemented and applied, leading to faster and more efficient algorithms for solving a broad class of optimization problems, especially in settings that require expensive function evaluations or simulations for performance estimation.
