Neuroscience is highly specialized?even visual submodalities such as motion, depth, form and color processing are often studied in isolation. One disadvantage of this isolation is that results from each subfield are not brought together to constrain common underlying neural circuitry. Yet, to understand the cortical computations that support vision, it is important to unify our fragmentary models that capture isolated insights across visual submodalities so that all relevant experimental and theoretical efforts can benefit from the most powerful and robust models that can be achieved. This proposal aims to take the first concrete step in that direction by unifying models of direction selectivity, binocular disparity selectivity and 3D motion selectivity (also known as motion-in-depth) to reveal circuits and understand computations from V1 to area MT. Motion in 3D inherently bridges visual submodalities, necessitating the integration of motion and binocular processing, and we are motivated by two recent paradigm-breaking physiological studies that have shown that area MT has a robust representation of 3D motion.
In Aim 1, we will create the first unified model and understanding of the relationship between pattern and 3D motion in MT.
In Aim 2, we will construct the first unified model of motion and disparity processing in MT.
In Aim 3, we will develop a large-scale biologically plausible model of these selectivities that represents realistic response distributions across an MT population. Having a population output that is complete enough to represent widely-used visual stimuli will amplify our ability to link to population read-out theories and to link to results from psychophysical studies of visual perception. Key elements of our approach are (1) an iterative loop between modeling and electrophysiological experiments; (2) building a set of shared models, stimuli, data and analysis tools in a cloud-based system that unifies efforts across labs, creating opportunities for deep collaboration between labs that specialize in relevant submodalities, and encouraging all interested scientists to contribute and benefit; (3) using model-driven experiments to answer open, inter-related questions that involve motion and binocular processing, including motion opponency, spatial integration, binocular integration and the timely problem of how 3D motion is represented in area MT; (4) unifying insights from filter-based models and conceptual, i.e., non-image- computable, models to generate the first large-scale spiking hierarchical circuits that predict and explain how correlated signals and noise are transformed across multiple cortical stages to carry out essential visual computations; and (5) carrying out novel simultaneous recordings across visual areas. This research also has potential long-term benefits in medicine and technology. It will build fundamental knowledge about functional cortical circuitry that someday may be useful for interpreting dysfunctions of the cortex or for helping biomedical engineers construct devices to interface to the brain. Insights gained from the visual cortex may also help to advance computer vision technology.
The processing of visual motion and depth information is essential for a wide variety of important human abilities, including navigating through the world, avoiding collisions, catching and grabbing objects and interpreting complex scenes. To understand how neurons in the visual cortex transform and represent the information that underlies these abilities, we aim to initiate the development of a more complete, biologically constrained and openly available computer model of motion and depth processing that will be used to guide, and to interpret and incorporate results from, primate visual neurophysiological and psychophysical experiments. Gaining an understanding of the normal function of cortical neural circuitry is an important step in building the fundamental knowledge that someday may help to improve the ability to assess dysfunctions of the cortex and may help bioengineers create devices that interface to cortical circuitry to treat disorders and overcome disabilities.
