Our goal is to understand how the brain accomplishes visual object recognition. Evidence obtained under this grant and from other labs suggests that each visual image is processed along the ventral visual cortical process- ing stream into a new pattern of neural activity at its top level -- the inferior temporal cortex (IT) -- that con- veys explicit information about object identity, even in the face of substantial view uncertainty (?invariance?). That IT population representation is thought to be causally responsible for object recognition. But precisely how does the IT population account for a seemingly in?nite number of object discriminations? What are the behaviorally critical ?features? conveyed by IT? How many? How can they be described? Here we aim to build and test image-to-IT-to-behavior models that are predictively accurate over the entire domain of core visual object recognition behavior. Substantial prior work argues that we should start by test- ing and developing the IT 100.1f model family: all models in that family state that IT conveys ~100, image- computable ?features? in its activity sampled at ~1 mm scale. How can we test and develop such models? First, this model family predicts that we can build and provide a single, low dimensional (<100) Euclidean em- bedding space to predict all basic and subordinate level object discrimination tasks (Aim 1). Second, the model family predicts that we can discover the particular aspects of IT activity (called IT ?features?) as those that, when weighted and summed, exactly predict behavioral object confusion of every image (Aim 2a). Third, the model family predicts that temporary suppression of individual, mm-scale portions of IT cortex will produce reliable, predictable patterns of behavioral disruption across all basic-level and subordinate level object tasks (Aim 3). Fourth, the model family posits that differences in IT neural tuning functions at spatial scales less than ~1 mm are irrelevant for core object discrimination behavior ? a prediction we will test with both record- ing (Aim 2a) and neural perturbation (Aim 3) experiments. Finally, the model family motivates our goal (Aim 2b) of characterizing the complete set of ~100 IT features with image-computable functions and with human shape adjectives. While substantial preliminary data support these predictions and goals, a complete model has not yet been built or tested. If these aims are accomplished, this work would transform our understanding by showing pre- cisely how core object recognition is causally accounted for at the level of IT cortex, and by providing a model that would accurately predict how any image manipulation or direct IT neural intervention would alter any core object recognition behavior.
Visual object recognition is fundamental to our well-being, yet we only have a crude understanding of the brain regions that underlie this ability. The goal of these experiments is to build and test mechanistically motivated models of how neural processing and activity in the brain underlies our object recognition abilities. The results will support an understanding of how our brain sees and evaluates objects, and is likely to give new insight into how to augment or repair our object recognition abilities.
Showing the most recent 10 out of 25 publications
