This project, aiming at accelerating research and development of computer interfaces that support natural conversational interaction with virtual humans, develops a virtual human toolkit for designing and experimenting multimedia programs that support real time conversational interaction with virtual humans, and resources for corpus-based research to generate animation of computer characters automatically from text and/or speech. The virtual human toolkit will consist of an integrated set of authoring tools and research tools for developing multimedia applications that incorporate virtual humans in intelligent tutoring systems, speech therapy systems, and other applications. The toolkit provides authoring tools and tutorials for presenting and interacting with media (text, images, and animations), designing and controlling lifelike 3D computer characters, and designing natural spoken dialogs. Research advances and real time systems should impact individuals and society by providing efficient and effective computer interfaces. The research tools can be used to gain new insights into the learning and therapeutic processes by designing systems that manipulate presentation of media and behaviors of virtual humans. The proposed work should lower the barriers and provide new tools to advance knowledge in some of the following areas: intelligent tutoring systems, spoken dialog systems, speech recognition, computer character animation, machine perception and learning, and human social dynamics.
Broader Impact: The development of virtual humans that behave like sensitive and effective reading teachers, speech therapists, medical helpers, and general assistants might impact quality of life for many illiterate, challenged, or disabled individuals. Computer-based learning and tutoring systems provide unprecedented opportunities for benefiting from accessible and inexpensive tutoring and therapy systems.
