There is great need for accessible and effective intelligent tutoring systems that can improve learning by children and adults. In this project, the PI will investigate the role of facial expressions and head movements produced by a lifelike animated character with the goal of indicating important information in a story or the emotions of the characters. The PI's hypothesis is that stories in which the virtual storyteller emulates natural head movements and emotions will lead to increased engagement by subjects, more positive ratings of the storyteller, and better comprehension of the stories. To validate the hypothesis, the PI will conduct experiments in which she manipulates the head movements and facial emotions of a virtual storyteller, so as to emphasize specific speech intervals or to provide emotional expression through the storyteller?s voice. She will analyze videotapes of subjects listening to and looking at the storyteller to assess how attentive they are, and she will also employ questionnaires to measure the subjects? impressions of how believable, credible and human-like the virtual storyteller appears to be. She will further ask subjects to summarize the stories they heard, and will then analyze the summaries both for comprehension and for recollection of emotional content. By these means the PI expects to lay the foundation for a powerful new experimental approach for investigating engaging and effective communication by lifelike animated characters through speech, head movements and facial expression of emotions, and to derive important new insights about how voice and facial expressions can be combined in optimal ways to enable pedagogical agents to provide more believable and effective communication experiences.
Broader Impacts: This exploratory research will provide new insights and knowledge about the role of face and head gestures, and facial displays of emotion, on engagement and learning in intelligent tutoring programs that use lifelike pedagogical agents. Project outcomes will inform the design of pedagogical agents that can produce more engaging and natural dialogs and narrations, which in turn will ultimately lead to more effective technology that benefits diverse learner groups across a wide range of application domains.
