Social and affective perception is the critical input that governs how we interact with others during everyday life. Consequently, having a model of the neurobiological basis of social and affective perception is critical for understanding the neural basis of human behavior. The overwhelming majority of our understanding of the neural basis of social and affective perception comes from studies done in artificial lab settings, which cannot capture the richness, complexity, and salience of real-world social interactions. This project aims to fill this gap in knowledge. To accomplish this goal, the researchers will record electrical brain activity from patients undergoing neurosurgical treatment for epilepsy. To determine the region of the brain responsible for their seizures, these patients are implanted with electrodes in various parts of their brain and then they spend 1-2 weeks in the hospital during which they interact with doctors, nurses, friend and family visitors, etc. This award will support research into using the recordings from their brains to understand how these patients perceive and understand the actions, emotions, and communication during these interactions on a moment-to-moment basis. The results of these studies have the potential to transform our understanding of social and affective perception by illuminating the neural basis of these processes during real life, meaningful interactions. The lack of models of the neural basis of natural, real world social and affective perception is a critical impediment to understanding these processes and ultimately a developing treatments for debilitating neurological and psychiatric disorders of social and affective perception, such as autism, post traumatic stress disorder, etc. In addition, through education, mentoring, and teaching, this award will provide an avenue for new researchers to take advantage of the rare and valuable opportunity for basic neuroscientific research provided by direct recordings from the human brain. This research is supported by the EHR Core Research Program, providing funding for fundamental research in STEM learning and learning environments, broadening participation in STEM, and STEM workforce development.
Models of social visual perception developed using unnatural stimuli often assume that neurons have unchanging response sensitivity and are organized into bottom-up hierarchies. While some recent models acknowledge the role of feedback, they remain simplistic with a relatively limited number of core systems and often neglect of the role of social context and dynamic prior knowledge. These models are unlikely to fully generalize to natural social vision where the system can rapidly and actively adapt its response to optimize processing of rich and complex natural visual input. The PI and colleagues will combine intracranial EEG (iEEG) recordings captured during long stretches of natural visual behavior with cutting-edge computer vision, machine learning, and statistical analyses to understand the neural basis of natural, real-world visual perception. The goal of their program of research is to develop the first fully ecologically validated models of social perception. The researchers will use recent advances in iEEG in combination with cutting-edge gaze tracking technology, video analysis tools, and big data statistical and machine learning tools to understand the rapid, complex neural information processing that occurs during real-world social vision. The project will involve decoding the spatiotemporal patterns of neural activity and reconstruct the expressive features of people they see at these different levels on a moment-to-moment basis. The multidisciplinary nature of this project provides an excellent environment for students and postdocs to be trained in computational methods, statistics, and neuroscience. Given the rapid advance of high-level computational and statistical methods in neuroscience, this multidisciplinary training is critical for modern neuroscientists. Enhanced understanding of the mechanisms involved in social cognition has implications for teaching and learning. For example, knowing more about how people form impressions of one another can inform teachers' abilities to recognize and respond to students and other stakeholders in educational settings.
This project is funded by Integrative Strategies for Understanding Neural and Cognitive Systems (NSF-NCS), a multidisciplinary program jointly supported by the Directorates for Computer and Information Science and Engineering (CISE), Education and Human Resources (EHR), Engineering (ENG), and Social, Behavioral, and Economic Sciences (SBE).
