Goal-driven autonomy (GDA) is a reflective model of reasoning about goals to control the focus of an agent's activities by dynamically resolving unexpected discrepancies in the world state, which frequently arise when solving tasks in complex environments. This project is motivated by two observations about GDA agents. First, to perform well, comprehensive GDA agents require substantial domain knowledge; however, few techniques have been investigated for learning this knowledge. Second, while existing GDA agents have demonstrated good performance in a variety of tasks, understanding and generalizing their successes has been hindered by a gap between the kinds of domains that these agents aim to model and the representations that they use; for instance, the bulk of current research on GDA agents assumes STRIPS representations of the agent's goals and actions.
This project aims to study GDA agents that are capable of learning expectations, explanations, and goals. This project aims to develop methods that enable creation of GDA agents that can autonomously act and learn to: (1) identify situations where discrepancies take place between what they expect and what actually has happened; (2) explain the discrepancy; (3) decide which goals to try to achieve as a result of these explanations; and (4) act to accomplish these goals. In this work, the objective of each agent is to maximize its expected return as defined in reinforcement learning. This approach fits naturally with the 4-step GDA cycle, facilitates studying properties about GDA using the well-defined reinforcement learning framework, and enables the adoption of representation formalisms such as stochastic policies (i.e., probability distributions of state-action pairs), which are naturally suited to represent GDA agent's actions in the domains that GDA agents aim to interact with. This project aims to develop representational methods that combine FOL (First Order Logic) literals and actions with probabilities as the basis to represent GDA elements.
The potential Broader Impact of this research is significant due to the potentially large and widespread applications of goal-driven autonomy. With the pervasive presence of autonomous computing devices and software, there is an increasingly pressing need for technology that enables systems to recognize discrepancies in what they expect from their 'worlds', diagnose them, and then adjust themselves. This is a ubiquitous problem in all areas of computer science. For example, in the general area of ambient intelligence, automated systems, such as an air quality control system, must monitor and control a variety of devices; it is very difficult, if not impossible, for a programmer to foresee all potential situations that such a system will encounter. Another example is cyber security where given the openness that characterizes current networks and the continuous integration of new technologies and services into them, it is not feasible to implement counter measures for all potential threats in advance; instead, an agent-based system must continuously monitor the overall network, learn and reason about expectations, and act autonomously when discrepancies are encountered.
This project includes a vigorous educational component. Specifically, it plans to (1) regularly involve undergraduate students in developing and testing carefully scoped components of the project; (2) create a course on adaptive and self-aware GDA agents that transcends traditional boundaries in courses on agents, reinforcement learning, and planning; and (3) create and disseminate testbeds for GDA agents that include not only the project's GDA agents but also simulations and agent-simulation interfaces. Creating and disseminating testbeds will help remediate the lack of systems and agent-simulation interfaces that has been a repeated stumbling block for teaching about GDA agents.
