This collaborative research investigates a new class of dialog-based, home robotic healthcare assistants to facilitate a new level of in-home, real-time care to elderly and depressed patients, providing lower total costs and higher quality of life. An emotive, physical avatar, called a companionbot, which possesses the ability to engage humans in a way that is unobtrusive and suspends disbelief will be built in this project. The companionbot will be an integration of human language technology, vision, other sensory processing and emotive robotic technology to proactively recognize and dialog with isolated and elderly patients suffering from depression. The companionbot will utilize proactive or companionable dialog based on the context with users suffering from depression. This will require the first multimodal integration of a user model, environment model, and temporal processing with spoken dialog understanding and generation to produce dynamic dialog and emotive interaction, beyond the traditional scripted dialog and emotion. Object recognition, facial expression recognition, and human activity recognition will augment natural language processing to provide current and historical context important to dynamic dialog.
A team of skilled researchers, assembled from the University of Colorado Boulder, University of Denver, CU Anschutz Medical Campus, and Boulder Language Technologies, will work together to achieve the project goals. The investigators will use the companionbots as a tool to run clinical trials to monitor and dialog with their partners to detect signs of physical and emotional deterioration. The companionbots can then notify remote caregivers, as necessary, provide warnings, reminders, life coaching and therapeutic dialog, extending independence and quality of life, and even saving lives. The other benefits of such a system include continuous, annotated data to improve doctor-patient interaction and analysis, real-time monitoring of mental state for remote healthcare providers and, ultimately, real-time intervention as part of a comprehensive treatment strategy.
In addition, this research will promote both STEM practice and research education at the graduate and the undergraduate levels of the affiliated institutions. The companionbots are ideal for teaching the next generation of engineers and scientists in critical emerging technologies, as they permit either a deep focus on specific topics or an interdisciplinary perspective while providing a simple high-level interface to manage everything else. Furthermore, the project will develop related educational material to support others and will provide public outreach to K-12 classes in the area.
