Historically, AI researchers have primarily focused on developing techniques that work well for pre-specified objectives that provide a useful measure of how well the techniques are working. This approach is perfectly sensible in a situation where the techniques are not yet ready to make their way out of the lab and into the world. However, as AI is now being broadly deployed in the world, more thought needs to be put into the methodologies for designing the objectives of AI systems. This is because our aim is no longer just to evaluate whether our other techniques are able to pursue a given objective well, but rather to actually have them do good in the world. Besides the AI system's objectives, we must also specify where one part of the system ends and another begins, as well as how it models the world. Generally, it is not possible or desirable to simply hand off the system to a customer (in the broad sense of the word) who then must somehow fill in these blanks. AI researchers need to be involved in this process because they understand how the system works and are able to provide algorithmic support for these decisions. But rigorous computational frameworks for these processes are lacking, and they are what this research aims to provide.
Specifically, existing research in artificial intelligence, mirroring frameworks in economics and other related fields, is built on a conception of AI systems as agents. It generally proceeds from the premise that each such agent has a well-defined identity over time, well-defined preferences over the different ways in which things may proceed, and well-defined beliefs about the world as it is and how it will develop over time. Typical research then concerns the design of algorithms under the assumption that all these aspects have already been specified (with the common exception of still needing to do some learning about the environment). However, as we design real AI systems, we in fact need to specify where the boundaries between one agent and another in the system lie, what objective functions these agents aim to maximize, and to some extent even what belief formation processes they use. The premise of this research is that as AI is being broadly deployed in the world, we need well-founded theories of, and methodologies and algorithms for, how to design preferences, identities, and beliefs. Doing so in a responsible fashion will require the development and rigorous evaluation of new techniques. The project will address these questions from a rigorous foundation in decision theory, game theory, social choice theory, mechanism design theory, and the algorithmic and computational aspects of these fields.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
