As robots become more prevalent, it is crucial to develop ways for people to collaborate with them. This proposal aims to create collaborative robots through a combination of communication, perception, and action. Existing approaches are typically tailored to specific applications; yet people want to talk to robots about everything they can see and do. To address such limitations, this project will create a unified framework to enable robots to communicate with people to learn their needs, plan how to achieve them, and then perceive and act in the world in order to meet those needs. The research will be demonstrated with robots that can assist with household tasks, such as cooking and cleaning, as well as in manufacturing settings, such as collaborative assembly. The project will expose many people to collaborative robotics through an internship program with local high schools, a regional robotics conference, and the Million Object Challenge.
This project will create a model, the Human-Robot Collaborative Partially Observable Markov Decision Process, that enables robots to 1) automatically acquire object-oriented models of objects in the physical world; 2) communicate with people to understand their needs and how to meet them; and 3) act to change the world in ways that meet people's needs. Creating a unified framework requires bridging gaps between different aspects of the robotic system. This project focuses on creating a single probabilistic graphical model to represent the robot's states and actions in a hierarchical framework, allowing the robot to make plans that take into account its own uncertainty and to communicate with a person about everything it can see and everything it can do. Focusing on collaboration leads to reformulations of traditional problems in computer vision, planning, and natural language understanding enabling the robot to collaborate in new and more natural ways.
