A fundamental goal of perceptual neuroscience is to understand the neuronal representations that underlie our remarkable ability to perceive, recognize, and remember visual objects. In humans and non-human primates, these representations are produced by processing along the ventral visual stream, and conveyed by patterns of neuronal activity in its highest level -- the monkey inferior temporal cortex (IT). The key computational problem the ventral stream solves is that it produces an IT neuronal representation of visual images that conveys selectivity for object identity and category, with tolerance ("invariance") to changes in object position, size, pose, illumination and clutter. Indeed, although the shape selectivity properties of the ventral stream have received much study, we know very little about the mechanisms that construct that tolerance. The goal of this proposal is a mechanistic understanding of how the ventral visual stream constructs the tolerant ("invariant") visual shape selectivity that underlies our object recognition abilities.
In Aim 1 we ask: does naturally-acquired temporally contiguous experience "instruct" the formation of tolerance in the ventral stream? We have recently discovered that the tolerance of IT neuronal shape selectivity can be strongly and rapidly sculpted by altered temporal contiguity of unsupervised visual object experience. In this aim, we will use a series of closely-related visual experience manipulations to systematically test and characterize the role of this plasticity in position, size, and pose tolerance learning. This will illuminate its role in instructing adult visual object representation, and set the stage for longer-term studies of how these powerful representations are assembled during early development.
In Aim 2 we will take a comparative approach to ask how object information is transformed across two ventral stream areas (V4 vs. IT)? Using the same monkeys, same task, and same visual stimuli, we will use neuronal population methods to ask: How is the tolerance of the IT representation changed from the V4 representation? Is V4 shape selectivity preserved in the IT representation? Does the sparseness of visual representation change from V4 to IT? How does tolerant shape selectivity evolve in real time? Together, these experiments will inform a central question: "How is the tolerant object selectivity in IT built from earlier visual representation?", and the results will provide strong constraints on computational models of the ventral visual stream and guide our understanding of cortical information transformation more generally.

Public Health Relevance

Visual object recognition is fundamental to our well-being and our brain is remarkably good at solving this problem even though the same object can appear very differently to our eyes. The overarching goal of these experiments is a mechanistic understanding of how the visual system constructs the patterns of neuronal acitivity that solve this problem. This will lead to an understanding of the brain processes that allow us to see and evaluate the visual world (e.g. recognize and remember objects).

National Institute of Health (NIH)
National Eye Institute (NEI)
Research Project (R01)
Project #
Application #
Study Section
Central Visual Processing Study Section (CVP)
Program Officer
Steinmetz, Michael A
Project Start
Project End
Budget Start
Budget End
Support Year
Fiscal Year
Total Cost
Indirect Cost
Massachusetts Institute of Technology
Organized Research Units
United States
Zip Code
Yamins, Daniel L K; Hong, Ha; Cadieu, Charles F et al. (2014) Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proc Natl Acad Sci U S A 111:8619-24
Issa, Elias B; Papanastassiou, Alex M; DiCarlo, James J (2013) Large-scale, high-resolution neurophysiological maps underlying FMRI of macaque temporal lobe. J Neurosci 33:15207-19
Baldassi, Carlo; Alemi-Neissi, Alireza; Pagan, Marino et al. (2013) Shape similarity, better than semantic membership, accounts for the structure of visual object representations in a population of monkey inferotemporal neurons. PLoS Comput Biol 9:e1003167
DiCarlo, James J; Zoccolan, Davide; Rust, Nicole C (2012) How does the brain solve visual object recognition? Neuron 73:415-34
Rust, Nicole C; Dicarlo, James J (2010) Selectivity and tolerance ("invariance") both increase as visual information propagates from cortical area V4 to IT. J Neurosci 30:12978-95
Li, Nuo; DiCarlo, James J (2010) Unsupervised natural visual experience rapidly reshapes size-invariant object representation in inferior temporal cortex. Neuron 67:1062-75
Pinto, Nicolas; Doukhan, David; DiCarlo, James J et al. (2009) A high-throughput screening approach to discovering good forms of biologically inspired visual representation. PLoS Comput Biol 5:e1000579
Li, Nuo; Cox, David D; Zoccolan, Davide et al. (2009) What response properties do individual neurons need to underlie position and clutter "invariant" object recognition? J Neurophysiol 102:360-76
Li, Nuo; DiCarlo, James J (2008) Unsupervised natural experience rapidly alters invariant object representation in visual cortex. Science 321:1502-7
Cox, David D; DiCarlo, James J (2008) Does learned shape selectivity in inferior temporal cortex automatically generalize across retinal position? J Neurosci 28:10045-55

Showing the most recent 10 out of 15 publications