The University of Southern California's Institute for Creative Technologies and the Museum of Science, Boston will create life-sized, 3-D Virtual Humans that will interact with visitors as interpretive guides and learning facilitators at science exhibits. Through the use of advanced artificial intelligence and intelligent tutoring techniques, Virtual Humans will provide a highly responsive functionality in their dialogue interpretation that will generate sophisticated interaction with visitors about the STEM content related to the exhibit. The project exemplifies how the confluence of science, technology, engineering, mathematics and education can creatively and collaboratively advance new tools and learning processes. The Virtual Human project will begin to present to the visitor a compelling, real life, interactive example of the future and of the related convergence of various interdisciplinary trends in technology, such as natural language voice recognition, mixed reality environments, para-holographic display, visitor recognition and prior activity recall, artificial intelligence, and other interdisciplinary trends. The 3-D, life-sized Virtual Humans will serve as museum educators in four capacities: 1) as a natural language dialogue-based interactive guide that can suggest exhibits to explore in specific galleries and answer questions about particular STEM content areas, such as computer science; 2) as a coach to help visitors understand and use particular interactive exhibits; 3) be the core focus of the Science behind the Virtual Humans exhibit; and 4) serve as an ongoing research effort to improve human and virtual human interactions at increasingly sophisticated levels of complexity. The deliverables will be designed to build upon visitor experiences and stimulate inquiry. A living lab enables visitors to become part of the research and development process. The project website will introduce visitors to the technologies used to build virtual humans and the research behind their implementation. The site will be augmented with videos and simulations and will generate user created content on virtual human characters. Project evaluation and research will collect language and behavioral data from visitors to inform the improvement of the virtual guide throughout the duration of the grant and to develop a database that directly supports other intelligent systems, and new interface design and development that will have broad impact across multiple fields.
project, the University of Southern California’s Institute for Creative Technologies (USC ICT) and the Museum of Science, Boston (MoS) created 3D virtual, computer-generated characters to serve as museum "educators" in four capacities: 1) as natural language dialogue-based interactive "guides" that could suggest exhibits to explore in a museum gallery and answer questions about particular STEM content areas, such as computer science; 2) as a "coach" to help visitors understand and use a particular interactive exhibit; 3) as the basis of a STEM-focused technology exhibit; and 4) as an ongoing research effort that capitalized on museum visitor participation. A well-informed guide or interpreter can have a tremendous influence on the quality of a museum visitor’s experience. Unfortunately, human museum guides are often in short supply. This project sought to augment human museum interpreters with virtual ones, and determine whether virtual humans could create the enthusiasm, engagement and knowledge transfer that skilled human guides provide. The project created two installations of virtual humans, the Twins and Coach Mike. The Twins were a pair of near-photoreal virtual humans installed at Cahners ComputerPlace at the Museum of Science. They interacted in natural language using speech and could answer visitors’ questions about STEM topics such as computer science, suggest things to see at the museum, and even describe how they themselves worked. To understand visitors' questions, the characters used automatic speech recognition and a sophisticated statistical classifier to identify the best response to the question. To further the educational value of the Twins, an additional exhibit was placed on the side of the kiosk housing the characters called "The Science Behind Virtual Humans." That exhibit showed the programs that controlled the Twins on a computer screen, providing a dynamic window into the results of speech recognition and the statistical classifier. Statistical approaches to artificial intelligence, which were used in the Twins, benefit from access to interaction data. Data gathered from the large number of actual interactions between guests and the virtual Twins, was used to improve the performance of the system over the life of the exhibit. The Twins exhibit opened in December 2009. Anecdotal reports from museum staff indicated a very positive reception to the Twins. In 2011, a formal summative evaluation of the project was performed by the Institute for Learning Innovations. The study involved observation of visitors, surveys and interviews with visitors. The four intended impacts for the Twins were that children (ages 7-14) and adults would: Increase their awareness about computer science and technology; Increase their engagement and interest in computer science and technology; Increase their knowledge about computer science and technology; Gain a positive attitude about computer science and technology. The study showed that all of the impacts were achieved and that all of the indicators for the impacts were achieved except for one (see http://informalscience.org/evaluation/show/628 ). Our second effort, Coach Mike, added a virtual human to an existing exhibit at the museum, Robot Park. At Robot Park, visitors could program a robot vehicle by arranging a set of instruction blocks into a program sequence. Coach Mike could introduce Robot Park to visitors, assist them with their programming task, help them debug their programs if they didn’t work the first time, and pose new challenges for them to try. Coach Mike opened in the spring of 2011 and was part of the summative evaluation that occurred in the summer of that year. A controlled study (Robot Park with and without Coach Mike) showed that Coach Mike was partially successful in achieving its intended impacts, which were the same as the Twins. Specifically, because Robot Park was already a highly successful exhibit, a ceiling effect was found in terms of impact on awareness, knowledge, and attitudes. However, Coach Mike’s presence did lead to significantly longer holding times, acceptance of programming challenges, and less misuse of the exhibit. A second study was performed in the fall of 2011 by the MOS evaluation team that focused on the impact of Coach Mike’s behaviors on visitors’ engagement with Robot Park. This experiment compared two different versions of Coach Mike: an enthusiastic, supportive version versus one that was "all business" and with less use of engaging features. Researchers found that the use of enthusiasm and self-regulatory feedback led to greater self-efficacy for programming. We estimate that over 200,000 people interacted with the Twins at the Museum of Science, with a somewhat smaller number for Coach Mike since it was launched some months after the Twins. In addition to the museum exhibition, the Twins were also exhibited as part of the NSF exhibit at the 2010 AAAS Conference in San Diego, and the Twins and Coach Mike were exhibited as part of the NSF exhibit at the 2012 USA Science and Engineering Festival in Washington, DC.
