The objective of this research is to develop new neural network structures to solve optimal control problems with dynamic decision making. These problems are quite complex since the system dynamics could switch modes at unknown times based on event based decision making. The approach is to develop the decision-making paradigms from cognitive science principles but their mathematical representations will use Decision Field Theory. Their solutions contained in neural networks will interact with another set of networks that embed solutions to the related optimal control problem formulated in an approximate dynamic programming framework.
Intellectual Merit
This research seeks to find unified controller solutions to problems which have both continuous and discrete elements in them. It is expected that the mathematical cognitive science ideas developed will lead to new representations and problem solving structures in computational neuroscience and control. The work proposed in this effort seeks to accomplish these objectives by offering a transformative approach that integrates concepts from system science and cognitive science.
Broader Impact
Abstractions and solution structures developed through this research can be used in consequence or emergency management systems like managing the aftermath of an earthquake, retrieving an impaired aircraft to stability and sustainable motion and landing, and managing multiple assets and allocation in striking responses to threats. Decision making structures resulting from this research can make tremendous impact on human-machine interactions too. For example, driver aid systems can be developed to augment human perception and enhance their cognition when they drive under impaired conditions.
This project was proposed to develop new neural network structures to solve optimal conrol problems with dynamic decision making; these problems are quite complex since the dynamics could switch modes at unknown times based on certain conditions. Outcomes from this project include new neural network based techniques which can produce optimal times for autonomous switched systems(no control). For example, paper published with this method investiagtes how to optimally switch drug regimen for a patient with HIV. Besides, the solutions for autonomou ssytems, this research has also produced optimal control for linear and nonlinear non-autonomous switched sytems(with control). A broader impact of the switched controller is that it could be used in hybrid vehicles where the sytem switches between different modes of operation to minimize gas usage. another application is with aircrafts which change their profiles(morphing aircrafts) depending on the flight conditions. Control solution structures have compact representations since they represent neural network weights; they are are comprehensive isnce they are obtained through 'adaptive critic ' based formulations solving dynamic programming equations. They are therefore,feedback controllers even though they are synthesized offline. This project has advanced the state of the art in neural networks since such class of problems has not been tackled with neural network formulations; it has also advanced state of the art in control of switched systems since such solutions were not available before for nonlinear systems and in such a compact form. Advantage of the neural network approach is that it is easily implementable. Some other outcomes from this research also merit attention: a global optimaization method has been developed by converting the optimization problem to an optimal control problem and solving it with an 'adaptive' criic based formulation. A new finite-horizon nonlinear optimal control technique which contains multistep solutions in a single network and does not need any approximations to underlying equations as done by some in the existing literature has also been developed. All these algorithms have been devloped with proofs of convergence. Another outcome of this project is the modified Grover algorithm which can be classified as a reinfocement learning algorithm for optimization. This algorithm performs a lot more efefctively as compared teh commonly used soft-max method and in decision making. Consequently, this research will have a major impact on many application areas that involve individual or team decision making. A major conclusion from this project is that cognition based approaches offer good value and are worth exploring further in finding effective and optimal solutions for control and decision making in probabilistic and stochastic environment.
