An understanding of how the brain processes visual stimuli is confounded by the complexity of the natural visual world, combined with the intricate neuronal processing that occurs within and across multiple cortical areas. Over the last few decades, substantial progress has been made by using simple laboratory stimuli, such as moving bars or dots, to develop simple descriptions of neuronal tuning to these elements. This approach has provided a reasonable functional description of lower cortical areas, but it is unlikely to be sufficient to characterize the regions of the extrastriate cortex whose responses are thought to represent elements particular to the natural visual world. Though a joint Canadian-US collaboration, this project couples new experimental approaches based on a set of complex stimuli approaching natural vision with appropriately complex models, in order to understand how neurons in successive stages of cortical processing are tuned to more natural visual features.
Most previous work with natural stimuli has focused on the ventral pathway in the cortex, which is concerned with computing object shape and identity. This is an extremely challenging problem, as the dimensionality of shape space is unknown. This project focuses instead on the dorsal stream of the primate visual cortex, which is primarily identified with motion processing. The advantage of this approach is that motion, particularly that seen in natural vision, can be locally decomposed into a low-dimensional optic flow space that can be sampled using naturalistic stimuli designed for this proposal. The development of such stimuli will extend both the spatial and temporal complexity of probes to areas in the dorsal stream, while providing the necessary constraints for a novel nonlinear modeling framework that will be developed. These models will then be applied to motion stimuli derived through simulation of natural three-dimensional virtual environments, allowing the complex processing uncovered to be linked to natural visual features. Furthermore, by performing this study across successive areas comprising the dorsal hierarchy (V1, MT, and MST), this project aims to expose general principles of cortical processing, namely how higher level abstractions are derived from lower-level visual features.
This project is jointly funded by Collaborative Research in Computational Neuroscience and the OISE Americas program. A companion project is being funded by the Canadian Institutes of Health Research.
Our visual system can instantly parse complex visual scenes into objects, and their spatial locations and relative motions. It accomplishes this task though processing over a hierarchy of visual areas, developing selectivity to increasingly complex elements of the scene at each level. Understanding how this is accomplished is confounded by both the complexity of natural scenes, as well as the difficulty isolating the computations performed at each stage of processing. We studied this problem using a tightly integrated collaboration between neurophysiology and computational modeling in three cortical areas along the primate dorsal stream (primary visual cortex or V1, area MT, and area MST), which extract and process motion information. Targeting the dorsal stream allowed for the design of novel visual stimuli that could be modeled using new statistical approaches, and ultimately led to a deep understanding of the types were built of natural motion patterns, or "optic flow", which could drive each visual area, and allow for isolation of the computations being performed in each area. The resulting work has lead to insights into how selectivity to complex features is derived in the cortex from sensitivity to simple features in previous areas, as well as specific characterizations in each area [see attached image]. It also lead to the development of new statistical modeling frameworks that can be used to characterize nonlinear properties of neurons in a variety of sensory areas. Such work has thus provided new insights into how the cortex functions, as well as described new general approaches for further study in this and other systems. It has lead to five publications in prestigious journals, documenting both the computations being performed in visual area, as well as the novel modeling approaches. We have also shared the data generated from this work on a public database, and also made the developed modeling tools available. Furthermore, in producing state-of-the-art models of neurons in the dorsal stream, we have discovered the interaction between stimulus processing and behavior relevant to visual processing in awake animals, such as visual tasks and attention, which is the foundation for future work linking sensory processing and behavior. Over the project duration, three graduate students have been engaged and trained in the multi-disciplinary approaches: one has graduated and is a postdoc in computational neuroscience, and two are close to graduation, with strong publication records and a goal to pursue work in the field. They all have received multi-disciplinary training as well as international research experiment, in particular for the US student to participate in experiments at McGill University (Montreal, Canada). In addition to disseminating this work through publication in journals and speaking widely at conferences, the modeling components of the projects are now taught as a part of a computational neuroscience course at University of Maryland, and were also included in a Fall Computational Neuroscience at University of Goettingen (Germany). Finally, the work established a longer-standing collaboration between the two laboratories, the experimental lab of Chris Pack in McGill (Montreal, Canada) and the computational neuroscience lab of Dan Butts at University of Maryland, which is now investigating the relationship between this sensory processing and behavior, using the funded work as a foundation.
