Embedded systems of collaborating agents that are capable of interacting with their environments are becoming ubiquitous. These systems must be able to adapt to the dynamic and uncertain characteristics of an open environment based on the priority of tasks, availability of resources, and availability of alternative ways of satisfying these tasks, as well as tasks expected in the future. The project is developing a framework and supporting algorithms for multi-agent meta-level control, which will determine when adaptation should be done and how much effort should be invested in adaptation as opposed to continuing with the current action plan. In particular, the meta-level framework will support coordinating decentralized Markov Decision Processes and views this coordination as a global optimization problem that bootstraps from individual agent learning and vice-versa. The framework will be demonstrated in a real-world multi-agent tornado tracking application called NetRads.
A cooperative multiagent system (MAS) consists of a group of autonomous software systems/agents that interact with one another in order to optimize a global performance measure. These systems are finding applications in a wide variety of domains, including sensor networks, robotics, collaborative decision making systems and distributed control. It is crucial that these systems adapt to the dynamics of open environments. Metareasoning is the process of dynamically determining in real-time when this adaptation process should be done and how much effort should be invested in adaptation as opposed to continuing with the current action. As part of this project, we argue that equipping individual agents with multiagent metareasoning (MMLC) capabilities improves the performance of MAS. This project is a previously unexplored vein of inquiry to develop MMLC algorithms that address issues of scalability, partial information and complex interactions across agent boundaries in real-world domains. We evaluate these algorithms in the context of NetRads, a MAS consisting of a network of adaptive radars. The NetRads radar is designed to quickly detect low-lying meteorological phenomena such as tornadoes while the MCC agent manages multiple radars simultaneously. Netrads has a dynamically evolving environment (different types of weather phenomena are occurring unexpectedly) and agents have only limited partial views of the whole system. The goal of NetRads is to maximize the overall performance of the MAS to correctly track and identify weather systems. The main outcome of the project is a generalized framework for MMLC that reasons about the agent’s own actions and information it receives about other agents via inter-agent communication.. The project produced the following results: 1) Development of a mathematical model that facilitates agent coordination at the meta-level to efficiently support inter-agent interactions and reorganize the underlying network when needed. 2) developed unsupervised machine learning algorithms to learn meta-level polices on their own while incorporating heuristic rules to resolve conflicts among agent polices locally at both learning and execution stages; 3) we were able to efficiently decrease the exploration costs involved in searching for a solution by effectively using abstractions; 4) we leveraged the significance of shared tasks in the Netrads domain and the use of a reward function for the machine-learning perspective so that agent’s can determine values of tasks they need to perform from a partially global perspective 5) an empirical evaluation that spans both homogeneous and heterogeneous (more complex) environments to show that the adaptive approach helps to resolve conflicts and improve the overall performance. 6) we have also developed algorithms for agent coordination and organization in scale-free networks (an online social network like facebook is a scale-free network). The intellectual merits include 1) developed novel methods for multi-agent metareasoning and coordination by interleaving machine learning and conflict resolution in MAS 2) studied network (re)-organization problem and as a metareasoning problem 3) showed the importance of metareasoning in a multi-agent application. The following are the broader umpacts. Results from this project will be useful for improving performance in real-world applications, such as the sensor networks. It can also have significant impact on the next generation of embedded systems requiring metareasoning to operate effectively in open environments. The multiagent coordination techniques developed in this project will help establish their significance in distributed systems like meteorological control and multirobot systems as well as defense, commercial and civil applications. This project has provided an excellent opportunity for Ph.D. and undergraduate student research. The project resulted in multiple top-tier publications co-authored by graduate students. One doctoral student completed his dissertation on the topic and is currently a research scientist in a local company and the other student will begin a faculty position upon graduation. Two undergraduate students participated in the project as REUs. This was their first foray into funded research in computer science and artificial intelligence. They learned to identify project goals that suited their research interests, do related literature study and work collaboratively to design a solution to the problem of modeling reasoning options that support qualitative decision theory. The REU participation also enabled both students to enroll in the NSF funded Leaders in Computing program and participate in the Girls are I.T. program that focuses on creating an interest for STEM for girls at a local middle school.
