This project, developing an instrument to enable a pioneering way to canonically represent and systematically quantify behavioral manifestations for a suite of neuropsychiatric disorders, integrates a powerful computer cluster with various sensor modalities and associated immersive technology components that facilitate data analysis and perception of physical and virtual experiences. The instrument will bring together neuropsychiatrists and engineering scientists seeking the development and deployment of tools that promote discovery and new knowledge in their respective domains, and innovation with immediate societal impact. It aims to enable medical experts to explore clinical hypotheses during diagnosis and treatment. Concurrently, scientists/engineers will explore the implementation of new computational tools and algorithms that satisfy new requirements utilizing a set of clinical hypotheses.
Challenging the current understanding of a spectrum of neuropsychiatric conditions and triggering the explorations of new ways to affect them, this instrument should contribute to our understanding of the presence and evolution of mental states. Its unique characteristics will help in integrating a range of informational cues (e.g., visual, acoustic, haptic) along with virtual simulated information. This interplay between physical and virtual experiences and stimuli and their perception in an immersive environment constitutes the ultimate objective which will unleash creativity in better understanding and treat mental illnesses.
The tight coupling of hardware and software components will enhance the various types of interactions such as expert-patient, and patient-virtual objects. Specifically, the instrument will support the following research efforts: - Comprehensive compilation of behavioral data, - Development of visualization tools, - Delivery of haptic information feedback, - Creation and interaction of virtual objects, and - Enhanced sentiment understanding from audio/visual signals, facial expression representation and analysis. The instrument enables the creation of communication channels with clinics across the country. Moreover, it contributes in forming a strong foundation to explore and understand a broad variety of neuropsychiatric disorders manifested via verbal and non-verbal communication pathways, aims to form a foundation that promotes strong collaboration targeting real-world challenges with immediate societal and economic impact, and offers an environment for discovery via the strong coupling of data visualization, User Interfaces-User eXperience (UI-UX) design, immersive perception, data exploration, and machine learning.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
