The proposed research is to develop feedback control theoretic methods for learning and adaptation in multiagent systems and to investigate their potential as analysis and design tools for distributed adaptive systems. The proposed work stems from recent innovations at UCLA on the role of feedback control in the game theoretic framework of distributed learning. This recent work has overcome long standing perceived obstacles in game theoretic learning and opens new opportunities for distributed system design. The proposed research will emphasize underlying mathematical models of multiagent learning. The research directions to be explored are: Advanced analysis of feedback control based learning in games: The introduction of feedback control methods in learning in games is very recent, and there are many important unresolved issues. Proposed topics of interest include strategic advantage of feedback control based methods, analysis of systems with heterogeneous learning algorithms, dynamic network interconnections, and supervised switching for learning. Continuum action spaces: The learning in games framework currently applies to learning among a finite set of choices. This direction involves learning with decisions over a continuum, such as dynamic resource allocation problems. Issues include new feedback control based approaches to distributed optimization, severely limited information structures, and complementary sharing of information structures. Multiagent games with state evolution: The learning in games framework uses a static game setup. The state variable is the state of learning, but not inherent states of subsystems or the environment. This direction considers learning for games with internal states, also known as Markov or stochastic games. Issues include changes of time-scale for expert selection, feedback control based Q-learning, and integrator action. Application to Evolvable Hardware: Evolvable hardware is an emerging area that uses notions inspired by biological evolution for the design of reconfigurable, self-organizing hardware. The proposed concepts on multiagent may be viewed as an engineered evolution with its blending of feedback control and multiagent learning. This project will use the evolvable hardware paradigm to illustrate and motivate the proposed research. Intellectual Merit: There is an extensive body of research in the area of multiple player games. In the case of non-zero-sum games, the concept of mixed strategy (i.e, randomized) Nash equilibrium, despite its central role, has received considerable scrutiny as to how players, through repeated interactions, would ever converge to a Nash equilibrium. Indeed there are long standing examples to the contrary. The innovative basis of the proposed research establishes that simple notions from feedback control can enable such convergence. Consequentially, this research opens many new possibilities into how to design distributed adaptive systems from a game theoretic viewpoint. Broader Impact: The proposed research will have a broader impact on societal applications and undergraduate student eduction. Societal: The distributed systems concept is relevant in a multitude of domains, both engineered and social. These include data networks, distributed robotics, traffic networks, distributed design, power grid infrastructure, and distributed computation, as well as economic exchange, social exchange, and political coalition dynamics. The proposed work addresses a fundamental component of the mathematical models of such systems and has potential implications in a variety of areas. Educational: Key activities to be undertaken are 1) Introducing 1-unit freshman courses on the concept of distributed systems and feedback control and their applications. This is through UCLA's Fiat Lux freshman seminar series, and 2) Developing an research test-bed based on evolvable hardware as a venue to incorporate undergraduate research-level participation through directed study electives. Of course, these activities are in addition to the usual education and training of graduate student researchers and dissemination of results to the research community.
