Intellectual Merit: The PI proposes to work on continuous-time versions of Adaptive Dynamic Programming (ADP), an emerging novel class of designs which offers hope of brain-like capabilities in the (approximately) optimal management of large complex systems subject to random disturbance, nonlinearity and a need for foresight. He will work on stability proofs, convergence analysis and new designs, together. Specific research goals involve neural network ADP for output feedback and nonlinear filtering, neurocontrol with entropy and nonstandard value functionals, model-based continuous time ADP and small-time-step approximate tuning for model-free continuous time ADP. Neural networks provide a way to approximate unknown nonlinear functions, needed as part of general ADP, with greater accuracy than traditional approximators for a limited number of parameters.
Broader Impacts: The PI plans to write a textbook on ADP in engineering, which is needed very greatly at the present time. The related books currently available are either collections of papers less accessible to the average student (such as the Handbook of Learning and Approximate Dynamic Programming, IEEE Press, 2004) or textbooks which focus on methods of reinforcement learning developed for psychology or artificial intelligence, which do not scale well to large engineering tasks. The PI will also insert the new designs/algorithms into the practical applications worked on at the Automation and Robotics Research Institute (ARRI), into undergraduate courses, and into his program for high school students in Hands-On Design and Learning for Intelligent Control Systems.
