For years, the robotics community has sought to enable robots to efficiently learn new skills from a human trainer. While motivated by the idea of robots that are easy to teach, there's been a lack of focus on several aspects essential to this goal. This project will advance the state of the art in robot Learning by Demonstration (LbD) by focusing the issues of learning from everyday human partners. LbD work has usually been evaluated with expert humans, typically the system designer. We will develop implementations and experiments on a humanoid social robot to address the following research questions: 1. In continuous embodied experience, how can a robot use cues from the environment and the person to segment its own learning examples? 2. What social cues does the human use to indicate salient aspects of the environment? What expressive mechanisms can the robot use to communicate relevant aspects of the learning state to the human partner? 3. Imitation learning is broader than the typical ?human-demo robot-repeat? LbD interaction. How can the robot use the human as an information source beyond generalizing a way to repeat motion trajectories? For example, it may be biased to interact with objects it?s seen the human interact with.
The aim of this research project has been to get closer toward the goal of enabling robots to learn new tasks and skills from everyday people. This project takes a Human-Robot Interaction (HRI) perspective on the problem of "Learning by Demonstration" (LbD). In this project we have made a few significant contributions. We have argued that the majority of research in the field of robot Learning by Demonstration neglects the key point that this interaction takes place within a social structure that can guide and constrain the learning problem. In the realm of Learning by Demonstration research, it is unusual for people to go the "final step" and test that their algorithms and interfaces are appropriate for non-expert humans. Experiments with humans teaching robots have been the cornerstone of this research agenda. Over the three years of this project, we have conducted three main research investigations into the nature of how a robot can learn from social partners. Experiment 1: Using a small upper torso humanoid robot, we first conducted a study looking at how people structure the learning process for this robot when it is tasked with learning about object affordances. This experiment compared the robot learning about objects by itself, to the situation where it learns with a human partner. We compared the learning performance of Support Vector Machine (SVM) classifiers in these two situations. One of the main contributions is the insights that our experiment gave into the nature of learning input from everyday people: They start with simple examples, group examples in coherent chunks, the amount of data provided for a particular object is proportional to complexity, and they focus on rare events. Due to these differences, we found that socially collected data sets and non-socially collected sets produce SVM classifiers with differing performance. Most importantly, social data sets do much better on predicting rare effects. Experiment 2: The LbD field has tended to focus on one of the highest forms of human social learning--imitation. In a second line of investigation, we note that humans and other animals use many other kinds of social learning mechanisms. We implemented a series of four increasingly complex social learning mechanisms. The mechanisms were drawn from the literature on human imitation and social learning. Our main insight is that robot learning tends to focus only on the most complicated mechanism (imitation), and our hypothesis was that other social learning mechanisms can also inform and bias the exploration of a learner. We implemented computational models of four mechanisms (stimulus enhancement, emulation, mimicking, and imitation) and evaluated the impact that each mechanism has on a learner. Our experiment showed that each provides a distinct benefit over individual/self exploration of an environment. And depending on the nature of the domain and the behavior of the social partner, each learning mechanism can provide better performance than the others. Thus, we conclude that a robot learner (like a human learner) needs all of these mechanisms in its repertoire and the challenge for future work is to determine the appropriate framework to allow these mechanisms to coexist. Experiment 3: In an effort to make the learning process more interactive, we have been investigating the use of Active Learning algorithms in an HRI setting. The Machine Learning community has focused on the optimality of algorithms with respect to the learner. Our work additionally sheds light on the optimality with respect to the human "oracle" that will be providing answers to the robot's queries. We ran a study that compared passive learning to fully active learning and two hybrid approaches. Our experiment showed that learning was more efficient with active learning (when robot makes queries rather than just accepting examples). People found the robot more intelligent and engaging when it asked questions. However, there were mixed results in which type of active learning strategy was preferred. There is some personal preference, but also the robot has to be sensitive to what constitutes a good question. Finally, the notion of balance of control seems important. The majority of people in our study preferred to be in control of when the robot could ask questions. We have made an effort for our work to have a broader impact on our research community and the general public. We have made several demonstration to the research community over the past three years: at the AAAI Spring Symposia Series; at the AAAI Robotics Learning by Demonstration challenge in 2009, 2010, and 2011; and in the CHI 2010 Interactive Demonstrations. Additionally, we have demonstrated to the general public at a Transformers movie event organized by Dreamworks in the midtown Atlanta area. We have opened our lab to groups of more than 100 high school students during National Robotics Week for the past two years. Additionally, we have held tours for smaller groups of students a few times each year.
