Stochastic systems are those whose behavior is random and cannot be predicted accurately but can be analyzed statistically. Stochastic optimal control has a wide range of applications in robotics, space exploration, autonomous systems, finance, computational neuroscience and computational biology. Despite the long history of stochastic optimal control theory, existing methodologies suffer from limitations related to assumptions on the structure of the dynamics, the form of cost functions, and connections between control strategies and random disturbances. These assumptions have restricted the applicability of this control method to special classes of problems that typically have simpler descriptions. This project will expand the applicability of stochastic optimal control to a broader class of stochastic optimization problems. The educational benefits of this project involve development of a new course and instructional materials for advanced undergraduate and graduate students.
In particular, this research aims to develop novel and scalable stochastic control algorithms using the theory of forward-backward stochastic differential equation and their connections to probabilistic representations of solutions of backward nonlinear partial differential equations. To aid future research and adoption of this work into these domains, the code, data, and results developed during the course of this project will be distributed freely to the scientific community. The PIs plan is to integrate powerful methods on adaptive importance sampling and forward-backward stochastic differential equations to develop scalable iterative stochastic control algorithms. In addition, this research project plans to make generalizations and extensions of the theory of forward-backward stochastic differential equations to problems such as stochastic differential games, control-constrained and bang-bang stochastic control and stochastic control under non- smooth cost functions. The work on these generalizations involves the development of algorithms which, will further expand the applicability of stochastic optimal control into new domains and new tasks. The educational plan of this research project has several goals designed to engage undergraduate and graduate students in research and inspire students to work on challenging problems at the intersection of stochastic control and statistics. The educational benefits involve development of a new course and instructional materials for advanced undergraduate and graduate students.
