The goal of this project is to design and build gaze mechanisms for embodied agents that can achieve high-level social and communicative goals that people achieve using such mechanisms and that can be applied across a wide range of agent presentations and task domains.
Embodied agents promise significant social, cognitive, and organizational benefits through applications in education, training, rehabilitation, and collaborative work. However, in order to be effective across a wide range of applications, agents must be able to use the nonverbal social cues that humans employ in their communication and to employ these cues in whatever modality the agent is presented in - whether it be a social robot, a life-sized virtual human, or an animated avatar on a portable display. Gaze cues are particularly important social signals. Although they are subtle, they can serve as powerful mechanisms for achieving high-level social and communicative goals, such as improving a listener?s comprehension, controlling the flow of a conversation, and indicating interest in or appraisal of objects. This project investigates how such mechanisms might be designed and built for embodied agents and how similar social and communicative goals could be achieved using different agent representations across different task domains.
This investigation will involve (1) performing formal observational studies to better understand how people use gaze, (2) developing computational models that synthesize gaze behaviors that can be controlled precisely and retargeted to a range of agent platforms, and (3) evaluating in experimental studies the effectiveness of using gaze cues across a range of agent presentations and task contexts. Success in this project will create new knowledge on human gaze behaviors, connecting the high-level findings in the social science literature to more detailed, low-level cues and mechanisms. It will also produce a set of techniques that are based on this understanding for synthesizing controllable and flexible gaze movements that agents can use in order to achieve social and communicative goals. Finally, it will validate the effectiveness of the use of gaze cues by agents across a variety of agent presentations and task contexts.
A trans-disciplinary approach will combine rigorous, formal observational studies to build detailed models of human communicative mechanisms with practical efforts to build usable computational models that meet the needs of creating agents that work in real-world tasks. The focus on human models insures that the computational models are well founded, while the focus on developing practical algorithms guides the human studies towards creating understanding that will be most informative for agent design. The project plan involves connecting the disparate communities that work on developing social agents and training students to do the trans-disciplinary work required to create effective embodied agents. The project will also enable K-12 outreach efforts to use robots and connections to social science to engage students and increase participation by under-represented groups.
Embodied agents, including virtual characters and social robots, promise new interfaces, new products, and new applications in areas including education, training, and rehabilitation, such as a virtual coach that appears on a television screen and guides a patient through a physical therapy program. To successfully assist their users in these areas, agents must display natural, intuitive, and effective social behaviors. The goal of this project was to provide embodied agents with the ability to display social behaviors, particularly gaze behaviors, in the way that humans use them. To achieve this goal, the project team conducted detailed analyses of human gaze behaviors, used this analysis to develop computer algorithms that controlled agent behaviors, and tested whether or not these behaviors helped agents achieve more natural and effective interactions with their users. Following this process, the research team endowed various types of agents with sophisticated gaze capabilities to interact with users across a wide range of scenarios. For instance, the team developed a virtual teacher lecturing in history that gazed at a historical map in a way that improved its student’s recall of the locations on the map (see attached images for other examples). The algorithms that the research team developed to control agent behaviors and findings from validation studies have significantly improved scientific knowledge on the topic and practical methods for developing real-world applications that benefit society, resulting in a large number of scientific publications. The findings from the project have also been shared with the public through articles, video stories, and radio interviews in the popular press. The project provided three doctoral students in computer science with opportunities for advanced training and professional development, establishing the foundation for three doctoral dissertations in computer science. Several undergraduate students in computer science also engaged in project activities, gaining hands-on research experience and receiving academic mentoring. The research team carried out a number of activities that aimed at informing the public about embodied agents and research findings from the project, including several "open lab" events, multiple sessions of a two-day summer program that provided children and their grandparents with the opportunity to learn about and gain hands-on experience with social robotics, and public talks for K-12 groups and local communities.
