The overall goal of the research is to determine the nature of early vision's representation of the visual world, and the design principles of visual cortical function. The proposal centers on testing a hypothesis that embodies the efficient coding principle: that the representation of the visual world is based on extracting the local image statistics that are the most informative about natural images. That is, in order to make optimal use of its limited resources, the visual system omits an explicit representation of aspects of natural images that are predictable, so that it can concentrate its resources on what is not predictable, and therefore, informative. While this notion has been widely successful as an organizing principle for understanding retinal processing, its application to visual cortex has been much more limited, in part because of the complex structure of natural scenes. The planned approach overcomes this barrier, through the use of a novel mathematical strategy that recognizes the complex characteristics of the sensory environment. Implementing this approach leads to a range of specific (and occasionally surprising) predictions, owing in large part to the complex structure of natural scenes and their statistical regularities. If supported by the experimental results, the resulting view of early visual processing will broaden current notions of the calculations that neural circuitry performs, beyond simple filtering and feature extraction.
Aim 1 consists of three sets of experiments to test predictions of the hypothesis in a comprehensive way. Specifically, Aim 1A determines whether the distribution of local image statistics accounts for perceptual sensitivity to first, second, third, and fourth-order statistics.
Aim 1 B determines whether the scale-invariance of image statistics in natural scenes is mirrored by a scale-invariance of perceptual salience.
Aim 1 C extends the analysis from binary images (used in Aims 1A and 1B for simplicity) to gray-level images, and uses psychophysical results to predict as-yet unrecognized regularities in natural images. Since it is important to understand not only the overall goal of visual processing, but also how it is achieved, Aim 2 characterizes the computational mechanisms that underlie extraction of image statistics. Specifically, Aim 2A focuses on dynamics, and how these calculations evolve as the perception of an image advances in time from a """"""""gist"""""""" to a detailed appreciation.
Aim 2 B determines whether image statistics are calculated via a discrete set of mechanisms likely to be embodied in dedicated circuitry, vs. a continuum of virtual ones. Successful completion of this research will advance the understanding of the goal and design principles of cortical visual processing, and thus, will support the rational design of advanced therapeutic modalities, such as neural prosthetics for loss of visual function.

Public Health Relevance

To guide action and support perception, visual information from the retina must be interpreted by calculations carried out in the brain. The proposed research aims to advance the understanding of the initial stages of these calculations: what they are, and how they are carried out. Successful completion of this research will thus support the rational design of advanced therapeutic modalities, such as neural prosthetics for patients with visual loss.

National Institute of Health (NIH)
National Eye Institute (NEI)
Research Project (R01)
Project #
Application #
Study Section
Central Visual Processing Study Section (CVP)
Program Officer
Steinmetz, Michael A
Project Start
Project End
Budget Start
Budget End
Support Year
Fiscal Year
Total Cost
Indirect Cost
Weill Medical College of Cornell University
Schools of Medicine
New York
United States
Zip Code
Hu, Qin; Victor, Jonathan D (2016) Two-Dimensional Hermite Filters Simplify the Description of High-Order Statistics of Natural Images. Symmetry (Basel) 8:
Victor, Jonathan D; Thengone, Daniel J; Rizvi, Syed M et al. (2015) A perceptual space of local image statistics. Vision Res 117:117-35
Rucci, Michele; Victor, Jonathan D (2015) The unsteady eye: an information-processing stage, not a bug. Trends Neurosci 38:195-206
Nitzany, Eyal I; Victor, Jonathan D (2014) The statistics of local motion signals in naturalistic movies. J Vis 14:
Aytekin, Murat; Victor, Jonathan D; Rucci, Michele (2014) The visual input to the retina during natural head-free fixation. J Neurosci 34:12701-15
Hermundstad, Ann M; Briguglio, John J; Conte, Mary M et al. (2014) Variance predicts salience in central sensory processing. Elife 3:
Menda, Gil; Shamble, Paul S; Nitzany, Eyal I et al. (2014) Visual perception in the brain of a jumping spider. Curr Biol 24:2580-5
Nichols, Zachary; Nirenberg, Sheila; Victor, Jonathan (2013) Interacting linear and nonlinear characteristics produce population coding asymmetries between ON and OFF cells in the retina. J Neurosci 33:14958-73
Zaidi, Qasim; Victor, Jonathan; McDermott, Josh et al. (2013) Perceptual spaces: mathematical structures to neural mechanisms. J Neurosci 33:17597-602
Victor, Jonathan D; Thengone, Daniel J; Conte, Mary M (2013) Perception of second- and third-order orientation signals and their interactions. J Vis 13:21

Showing the most recent 10 out of 63 publications