The representation of a single object in the brain typically involves dozens of brain regions spanning the cortex. In each region, a population of neurons will represent a subset of that object's features, such as its color and shape in visual area 4. Yet, how all of these di erent feature representations are integrated to produce a single object representation is unknown. The question of how the brain performs this integration is a subpart of the binding problem known as the assignment problem (or property binding). In this project, we will use a combination of theoretical, correlational, and causal methods to discover how the visual system integrates features from both of the two, anatomically segregated visual processing streams: The dorsal where stream and the ventral what stream. This instantiation of the assignment problem is extremely important in day-to-day life. We will use two tasks: One that exploits the innate, untrained preference of humans and many animals for viewing novel images; and one that requires attention to be directed toward a particular, cued image, regardless of its novelty. Crucially, both tasks require stimulus representations from each of the two visual streams. As these tasks are performed, we will simultaneously record from dozens of neurons in four brain regions that we hypothesize are involved in the assignment process. Next, we will reversibly inactivate each of the four regions in sequence while monitoring behavior and neuronal activity. In parallel with these experiments, we will develop a neural network model that mirrors the conditions of assignment in the visual system { with two sensory populations, modeled after the dorsal and ventral visual streams, feeding into a third population that produces an estimate of the stimuli responsible for the sensory input. Our theoretical and experimental work will interact at every step. The model will make predictions for both behavior and neural dynamics in our two tasks, and will guide our experimental design; the behavioral and electrophysiological data we collect will provide constraints on our model and guidance as to which neural mechanisms are necessary for the brain's solution to the assignment problem. The results of this project will not only unify large bodies of visual and cognitive neuroscience that focus exclusively on one visual stream stream or the other, they will also provide crucial and highly novel insight into how the visual system solves the assignment problem. The solution will also make predictions for assignment in other modalities, such as assignment between visual and auditory stimulus representations. Finally, in vision and across other sensory modalities, the assignment process is essential to many behaviors and, when disrupted, can lead to numerous cognitive and sensory-motor de cits that often present as symptoms of brain diseases and conditions including Alzheimer's disease, autism, schizophrenia, stroke, and attention de cit disorder. By understanding how assignment is performed and used to guide actions, we will reveal avenues for treatment of these diseases and conditions as well as give insight into their underlying causes.
We will discover how stimulus features that are represented in disparate neuronal populations can be integrated with each other to guide exible behaviors. We will study this computation, known as the assignment problem, in the context of the two, what' and where,' visual processing streams using theoretical, correlational, and causal methods. Disruption of the assignment process can lead to many visual-motor and cognitive de cits; by determining how assignment is performed, we can better understand the pathologies that emerge from its disruption, and how to alleviate those pathologies through targeted intervention.
