Sequential decision making in the presence of uncertainty arises in problems that span transportation and logistics, energy, health and military operations. Real-world instances of these problems are fundamentally intractable, beyond the reach of our most powerful computers. Parallel research among communities such as operations research and computer science has produced a diversity of algorithmic strategies that offer unique features, but significant language and notational barriers have limited the sharing of ideas. This workshop will consist of a series of conversations so that leading professionals, post-docs and students from both communities will better learn the languages of both communities. A major activity will be the design of challenge problems that benefit from the combined skills of both communities. For example, the problem of optimizing fleets of UAVs is a problem class that would be solved in very different ways by each community. Computer scientists tend to solve these as swarms of loosely coordinated, independent agents. Operations researchers have the tools to optimize these fleets using large-scale optimization, reflecting the perspective of a single controller.
If successful, the workshop will lead to avenues for improved collaboration between the operations research and computer science communities, including a roadmap for future collaboration between the two research communities. Ultimately this should lead to increased capabilities for solving problems that have been intractable up to now by exploiting the unique skills that each community brings, most prominently the computational power of machine learning and mathematical programming.
The workshop consisted of equal numbers of participants from operations research and computer science, all working in the general area of stochastic optimization. The core theme of the course was to bring out differences in vocabularies, as well as the models and algorithms favored by each community. Prior to the workshop, we solicited everyone’s favorite definition of a state variable, primarily to bring out the lack of a consensus regarding a definition of one of the most fundamental concepts in stochastic optimization. The conference was then roughly organized around major classes of policies, including sampling-based approaches for lookahead policies, policies based on value function approximations, direct policy search, and closed with a discussion of multiagent optimization. A major difference between computer scientists (AI) and operations researchers was in the types of problems they worked on. The AI community works almost exclusively on discrete action spaces which are easy to enumerate, while the OR community primarily (but not exclusively) works on problems with high-dimensional, vector valued actions. For example, everyone in the OR community is familiar with powerful tools such as Cplex and Gurobi, but I had to be warned that the CS participants had generally not heard of these packages. Vector-valued actions arise often in a range of resource allocation problems, where we exploit convexity to develop efficient algorithms. The CS community, on the other hand, worked with discrete action spaces (think about traversing a transportation network with a car or robot), where convexity is not a property that can be exploited. An effort was made to bring the two communities together by posing a series of challenge problems that bring out issues that are familiar to different communities. For example, while the OR community is struggling with uncertainty along with their vector-valued action spaces, they tend to assume that the state of the system is known (that is, we know where the trucks are or how much blood is in inventory, but not the future demands). The OR community also tends to assume they know how to model the dynamics that describe how the system evolves from one time period to another. The AI community, on the other hand, is often interested in problems where the state of the system is not directly observable, and frequently works on problems where the dynamics of the problem are not known (known as "model free").
