PI: Sarkar N. & Warren, Z. Proposal Number: 0967170 This interdisciplinary project proposes fundamental, transformative technology improvements to assist children with Autism Spectrum Disorder (ASD). The objective is to develop a novel affect and attention sensitive virtual reality -based assistive technology for ASD intervention. Social communication and social information processing are thought to represent core domains of impairment in children with ASD. The proposed research will develop technologies capable of targeting individual deficit by flexibly and adaptively responding to subtle affective and attentive changes in individuals with ASD during social paradigms and create VR-based ASD intervention technology that can present itself as a realistic and powerful intervention platform. The individual, familial, and societal impact associated with ASD is enormous. Therefore, an important direction for research on ASD is the identification and development of technology that can make application of effective intensive treatment more accessible and cost effective. To address this need, this project will combine VR-based technology with affective computing using physiological signals and attention inference through eye gaze measurement to develop a new paradigm for autism intervention that will appreciably transform the ability to understand and tailor interventions to the specific vulnerabilities of children with ASD. The project will develop an assistive technology that will result in an affect and attention sensitive system for detecting, adaptively responding to, and optimizing levels of social interaction for children with ASD during VR interactions. The system is scalable and adaptable to a broad range of intervention strategies, providing flexibility in design and implementation.
Intellectual Merit: The proposed research advances the design and authoring of adaptive virtual environments for use with a challenged population of children. It has the potential to significantly contribute new computational methods for affective computing, particularly affective computing mediated by physiologic signal processing. Primarily, the project will develop new framework to design virtual environments for social interaction that intelligently combine both affective and attentive information into adaptive and controllable response systems. The proposed activity also represents a new technology that will fundamentally advance the engineering knowledge of human-computer interactions as well as the understanding of the mechanisms that underlie the presumed core social impairments, and associated interventions, with ASD.
Broader Impacts: Emerging research suggests as many as 1 in 150 children are diagnosed with an autism spectrum disorder and ASD related care costs the nation over $35 billion annually. The low-cost and highly deployable technologies developed in this proposal could have significant impact on this population. They may also create a completely new technological intervention methodology for children with ASD. This research may further a technology that can enable all core components of effective intervention at only a fraction of the cost of typical intervention programs, while at the same time increasing the ability of the intervention provider to systematically control and promote intervention related skills targeting individual deficit. The educational activities will train and mentor undergraduate and graduate students in the proposed research, and develop one module in a one-credit seminar course targeted to engineering freshman class. The outreach activities will include offering research opportunities to high school students, especially among groups currently underrepresented in STEM (science, technology, engineering, and mathematics) fields, and providing high school teachers with research experience during summer in the design and research evaluation of advanced technologies for autism intervention. Outreach objectives will also involve usability testing to elicit feedback from behavioral professionals and children with ASD. This project represents an opportunity to leverage a unique collaboration that brings cutting-edge engineering, psychology, and clinical knowledge to the development of a pragmatic and efficacious assistive intervention technology addressing core symptoms of ASD.
The Centers for Disease Control and Prevention estimates 1 in 68 children in the United States have an Autism Spectrum Disorder (ASD). ASD is associated with enormous individual, familial, and social costs across the lifespan. Many families and service systems struggle to provide appropriately intensive evidence-based interventions due to extreme resource limitations. As such, there is an urgent need for more efficacious treatments that can be delivered across resource-strained environments. This research project leverages a common preference that many children with ASD show towards technological engagement, to design a novel virtual reality (VR)-based adaptive social skill training system that more effectively addresses social communication deficits associated with ASD. The novel social skill training system uses a VR platform to present a social task (Fig. 1). The participant interacts with virtual avatars in social settings. The avatars are capable of dynamically changing their facial expressions (Fig.2) and making gestures (Fig. 3) based on the context of interaction. While the participant interacts with the avatars, his attentional and emotional state is inferred from his physiological responses (e.g., heart rate, skin sweating, muscle contraction etc.) using new machine learning algorithms. The participant’s eye gaze is continuously monitored in real-time to further evaluate engagement and attention as well as better understand how the individual processes social information within the VR platform. The VR task presentation is integrated with both emotion and engagement inference systems such that the social task can make autonomous adjustments based on not only on performance (e.g., correct or incorrect) but also on one’s emotional and attentional states (e.g., engaged or not engaged). The complete integrated platform works as closed-loop system (Fig. 4), adaptively changes task difficulty in an individualized manner to enhance learning. The social task training system was tested in several usability studies with teenagers with ASD as well as typically developing peers. These studies were designed to understand how individuals with ASD process faces and recognize emotion, how attention-based feedback helps in social communication, and how physiological responses during social tasks in VR can be utilized to understand the emotional states of the individuals with ASD. Specific findings indicate that i) the VR-based intervention system was acceptable to the target population; ii) there is a strong indication that real-time attention and emotion-based feedback can improve learning of social skills, and iii) emotion and face processing in VR with the help of eye gaze tracking and physiological response analysis can shed light on the atypical information processing mechanisms of children with ASD. The results are promising and indicate the value of such an intelligent VR-based system for potential intervention application. The grant offered research opportunities and training for both typical and underrepresented graduate and undergraduate students. It also provided research opportunities for high school teachers and high school students in STEM related disciplines. It fostered interdisciplinary research combining engineering and psychological sciences and provided opportunities to mentor early career scientists. The results of this research were disseminated to both scientific and lay communities to underscore the importance of such research endeavors. Ultimately, VR-based therapeutic tools that can target and specifically address impairing social communication processes associated with ASD may allow for more widely accessible intervention opportunities of specific impact.
