Real world scenes contain a wealth of information that guide where we look and help us search for things in our visual environment more efficiently. For example, if you were looking for a person in a city, you would look mostly on the sidewalk, whereas if you were looking for a car, you would concentrate your attention on the street. Despite the fact that behavioral experiments have increasingly quantified the role of object and scene knowledge in the guidance of attention and eye movements, models of these processes, particularly neural models, neglect the role of visual knowledge. The goal of this project is to determine whether regions of the brain shown to be important for object and scene recognition are involved in visual guidance in natural scenes. Prior results, including our preliminary data, show that the neural activity from object processing regions can be used to predict what object a person is going to look at next. However, critical questions that remain are: is this predictive activity influenced by scene and object knowledge and is it causally related to visual guidance? Answering these two questions are the specific goals of this proposal. Individuals undergoing neurosurgical evaluation for epilepsy provide the rare opportunity of recording directly from the human brain (intracranial electroencephalography, iEEG), which provides a superior spatial and temporal resolution measure of brain activity compared to other technique. These direct recordings also allow for electrical brain stimulation (EBS), which can provide causal evidence tying the activity in particular regions to cognitive function. Finally, these data will be supplemented by magnetoencephalography (MEG) data to examine whole brain effects in healthy individuals. iEEG and MEG data arising from regions involved in object and scene recognition will be analyzed by multivariate machine learning techniques to continually classify what subjects are viewing on a moment-to- moment basis. Furthermore, we will try to predict what object subjects will view next during free viewing and visual search in natural scenes based on their neural data. We will assess how these neural signals are modified by the presence or absence of information about typical locations of objects or people in the scene that have been shown to guide behavior. Finally, using EBS we will determine if there is a causal link between the activity in regions involved in coding for object and scene knowledge and visual guidance in natural scene vision. If successful, these studies would necessitate a substantial reshaping of models of visual attention in the human brain. The results could form the foundation of a program of research into the neural basis of attention and eye movement guidance in the real world. Attention, perception, and eye movement abnormalities are seen in a host of neurological and psychiatric disorders. Thus, these studies, and the models that arise from them, have the translational potential to advance our understanding of the neurological basis of these disorders and suggest potential neurally inspired rehabilitation strategies.
Abnormalities in visual attention and eye movement guidance are seen in a host of neurological and psychiatric disorders, including visual agnosia, neglect, schizophrenia, autism, etc. The neural basis of visual attention and eye movement guidance in the real world is unknown; therefore it remains unknown how neural abnormalities in these disorders relate to real world visual deficits, which is a critical barrier to designing hypothesis-driven remediation strategies. The proposed research will take critical early steps towards understanding the neural basis of real world attention and eye movement control, which has the potential to transform our understanding of these critical visual processes. The results of this research will lead to a deeper understanding of the neurobiology of eye movement and attentional guidance, a circuit relevant to a number of disorders, and take a critical step towards developing hypotheses for remediation strategies.
