The goal of this project is to test a novel theoretical framework for understanding how the ventral pathway subserves object vision. In the standard framework, a series of neural operations on 2D image data through many intermediate cortical stages, including area V4, leads to high-level perceptual representations, including representation of object identity, at the final stages of the ventral pathway. However, our preliminary microelectrode data from a fixating monkey show that many neurons in V4 represent volumetric (volume- enclosing) 3D shape, not 2D image patterns. These neurons respond to many different 2D images that convey the same 3D shape with different shape-in-depth cues, including shading, reflection, and refraction. They even respond preferentially to random dot stereograms that convey 3D volumetric shape with no 2D cues whatsoever. Moreover, our preliminary results with 2-photon functional imaging in anesthetized monkey V4 show that 3D shape signals are grouped by their similarity, and also group with isomorphic (same outline) 2D shape signals (which could contribute evidence to corresponding 3D shape inferences). We propose to capitalize on these preliminary data by demonstrating the prevalence of 3D shape tuning in area V4, analyzing the 3D shape coding strategies used by these neurons, and measuring how 2D and 3D shape signals are arranged at a microscopic level across the surface of V4. We expect these results to provide strong evidence that extraction of 3D shape fragments is a primary goal of V4 processing. This early extraction of 3D shape information, just two synapses beyond primary visual cortex, would suggest a competing framework for understanding the ventral pathway, in which the initial goal is to represent 3D physical structure, independent of the various 2D image cues used to infer it. In this framework, object recognition would be based on preceding information about 3D physical structure, which would explain why human object recognition is so robust to image changes, in a way that the best computational vision systems are not. The scientific impact of this work would be to divert vision experiments toward understanding representation of real world 3D structure (rather than 2D planar stimuli) and to encourage computational vision scientists to incorporate early 3D shape processing into the deep convolutional network models that are the current state of the art.

Public Health Relevance

This study will help explain how the human brain achieves such remarkably robust perception of visual objects. The results will help guide future rehabilitative and prosthetic strategies for patients with visual impairments, by elucidating neural strategies that could be used to bring computer vision prosthetics up to human performance levels, and by discovering specific neural signals for object information that could be replicated by electrode array implants in higher-level visual cortex.

Agency
National Institute of Health (NIH)
Institute
National Eye Institute (NEI)
Type
Research Project (R01)
Project #
5R01EY029420-02
Application #
9773064
Study Section
Mechanisms of Sensory, Perceptual, and Cognitive Processes Study Section (SPC)
Program Officer
Flanders, Martha C
Project Start
2018-09-01
Project End
2023-05-31
Budget Start
2019-06-01
Budget End
2020-05-31
Support Year
2
Fiscal Year
2019
Total Cost
Indirect Cost
Name
Johns Hopkins University
Department
Neurosciences
Type
Schools of Arts and Sciences
DUNS #
001910777
City
Baltimore
State
MD
Country
United States
Zip Code
21205