A fundamental goal of perceptual neuroscience is to understand the neuronal representations that underlie our remarkable ability to perceive, recognize, and remember visual objects. In humans and non-human primates, these representations are produced by processing along the ventral visual stream, and conveyed by patterns of neuronal activity in its highest level -- the monkey inferior temporal cortex (IT). The key computational problem the ventral stream solves is that it produces an IT neuronal representation of visual images that conveys selectivity for object identity and category, with tolerance (""""""""invariance"""""""") to changes in object position, size, pose, illumination and clutter. Indeed, although the shape selectivity properties of the ventral stream have received much study, we know very little about the mechanisms that construct that tolerance. The goal of this proposal is a mechanistic understanding of how the ventral visual stream constructs the tolerant (""""""""invariant"""""""") visual shape selectivity that underlies our object recognition abilities.
In Aim 1 we ask: does naturally-acquired temporally contiguous experience """"""""instruct"""""""" the formation of tolerance in the ventral stream? We have recently discovered that the tolerance of IT neuronal shape selectivity can be strongly and rapidly sculpted by altered temporal contiguity of unsupervised visual object experience. In this aim, we will use a series of closely-related visual experience manipulations to systematically test and characterize the role of this plasticity in position, size, and pose tolerance learning. This will illuminate its role in instructing adult visual object representation, and set the stage for longer-term studies of how these powerful representations are assembled during early development.
In Aim 2 we will take a comparative approach to ask how object information is transformed across two ventral stream areas (V4 vs. IT)? Using the same monkeys, same task, and same visual stimuli, we will use neuronal population methods to ask: How is the tolerance of the IT representation changed from the V4 representation? Is V4 shape selectivity preserved in the IT representation? Does the sparseness of visual representation change from V4 to IT? How does tolerant shape selectivity evolve in real time? Together, these experiments will inform a central question: """"""""How is the tolerant object selectivity in IT built from earlier visual representation?"""""""", and the results will provide strong constraints on computational models of the ventral visual stream and guide our understanding of cortical information transformation more generally.

Public Health Relevance

Visual object recognition is fundamental to our well-being and our brain is remarkably good at solving this problem even though the same object can appear very differently to our eyes. The overarching goal of these experiments is a mechanistic understanding of how the visual system constructs the patterns of neuronal acitivity that solve this problem. This will lead to an understanding of the brain processes that allow us to see and evaluate the visual world (e.g. recognize and remember objects).

National Institute of Health (NIH)
National Eye Institute (NEI)
Research Project (R01)
Project #
Application #
Study Section
Central Visual Processing Study Section (CVP)
Program Officer
Steinmetz, Michael A
Project Start
Project End
Budget Start
Budget End
Support Year
Fiscal Year
Total Cost
Indirect Cost
Massachusetts Institute of Technology
Organized Research Units
United States
Zip Code
Rajalingham, Rishi; Issa, Elias B; Bashivan, Pouya et al. (2018) Large-Scale, High-Resolution Comparison of the Core Visual Object Recognition Behavior of Humans, Monkeys, and State-of-the-Art Deep Artificial Neural Networks. J Neurosci 38:7255-7269
Hong, Ha; Yamins, Daniel L K; Majaj, Najib J et al. (2016) Explicit information for category-orthogonal object properties increases along the ventral stream. Nat Neurosci 19:613-22
Aparicio, Paul L; Issa, Elias B; DiCarlo, James J (2016) Neurophysiological Organization of the Middle Face Patch in Macaque Inferior Temporal Cortex. J Neurosci 36:12729-12745
Rajalingham, Rishi; Schmidt, Kailyn; DiCarlo, James J (2015) Comparison of Object Recognition Behavior in Human and Monkey. J Neurosci 35:12127-36
Afraz, Arash; Boyden, Edward S; DiCarlo, James J (2015) Optogenetic and pharmacological suppression of spatial clusters of face neurons reveal their causal role in face gender discrimination. Proc Natl Acad Sci U S A 112:6730-5
Majaj, Najib J; Hong, Ha; Solomon, Ethan A et al. (2015) Simple Learned Weighted Sums of Inferior Temporal Neuronal Firing Rates Accurately Predict Human Core Object Recognition Performance. J Neurosci 35:13402-18
Cadieu, Charles F; Hong, Ha; Yamins, Daniel L K et al. (2014) Deep neural networks rival the representation of primate IT cortex for core visual object recognition. PLoS Comput Biol 10:e1003963
Yamins, Daniel L K; Hong, Ha; Cadieu, Charles F et al. (2014) Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proc Natl Acad Sci U S A 111:8619-24
Baldassi, Carlo; Alemi-Neissi, Alireza; Pagan, Marino et al. (2013) Shape similarity, better than semantic membership, accounts for the structure of visual object representations in a population of monkey inferotemporal neurons. PLoS Comput Biol 9:e1003167
Issa, Elias B; Papanastassiou, Alex M; DiCarlo, James J (2013) Large-scale, high-resolution neurophysiological maps underlying FMRI of macaque temporal lobe. J Neurosci 33:15207-19

Showing the most recent 10 out of 25 publications