This project will investigate a new approach to learning models of dynamical systems for use in advanced reinforcement learning architectures. It will develop a method by which a reinforcement learning system can learn multiple time scale models and use them as the basis for hierarchical learning and planning. the project(s objectives are to develop the mathematical theory of this approach, to examine its relationship to control theory and to behavioral and neural models of learning, and to demonstrate its effectiveness in a number of simulated learning tasks. The significance of the project for engineeirng is that the use of TD models will be major generalization of RL architectures, making them much more widely applicable. It also has the possibility of establshing the utility of TD modesl for system identification in more conventional adaptive control. The project also has implication for our understanding of animal learning able to model indirect and direct associations and their interactions in a mathematically principled way. An Additinoal impact of this research will be t o strengthen links between engineering, artifical intellegence, and biological studies of learning, thereby contributng to all three areas by facilitating a transfer of concepts and methods.
