Our actions can determine visual input by changing the state of objects in the environment. Because future states of an object are often predictable based on its current state and a planned action, such actions provide a rich source of perceptual expectation and may play an important role in perception by inducing plasticity in the visual system via predictive coding. This prediction of future object states based on actions is minimized in typical experiments on visual object perception, in which observers have no power to control the stimuli presented. To determine how intentional action influences predictive coding in visual cortex, we propose to develop novel experimental paradigms with which we will directly test three independently motivated but related hypotheses.
In Aim 1, we will test whether associative learning in a laboratory setting allows the brain to predict what an object will look lke after an action. In particular, we will examine whether neural responses for predicted object states in visual cortex are attenuated relative to unpredicted states.
In Aim 2, we will test whether associations between complex actions and real-world objects learned over a lifetime induce similar predictive coding in visual cortex. These same participants will also learn action-outcome associations for novel stimuli to bridge with the first aim, but will be trained extensivel to consolidate the associations in long- term memory.
In Aim 3, we will test where these predictions come from in the brain, using functional connectivity to explore mechanisms of prediction based on the timescale of memory involved. We will test the hypothesis that connectivity with visual cortex depends on whether action-outcome associations were recently or gradually learned. By dissociating these types of memory, we seek to isolate two sources of predictions: the hippocampus, which is specialized for the rapid extraction of information, and the neocortex, which comes to represent this knowledge more gradually. Building on the conceptual frameworks of object files and predictive coding, these studies will allow us to identify the mechanisms involved in learning the visual consequences of actions, and to develop a neural model of how actions influence visual processing via connections with complementary learning systems. The findings will help us to understand plasticity in visual pathways both in the normal brain, as well as during recovery and rehabilitation from deprivation and brain damage.
The proposed research tests how actions induce plasticity in visual cortex via predictive coding. Identifying the mechanisms by which we learn the visual consequences of our actions has relevance to understanding how the visual system recovers from deprivation and damage. The tasks that we develop may also help in the diagnosis of sensory deficits in disorders like schizophrenia, which has been linked to disruptions in the timing of predictive signals such as corollary discharge.
| Hindy, Nicholas C; Turk-Browne, Nicholas B (2016) Action-Based Learning of Multistate Objects in the Medial Temporal Lobe. Cereb Cortex 26:1853-1865 |
| Hindy, Nicholas C; Ng, Felicia Y; Turk-Browne, Nicholas B (2016) Linking pattern completion in the hippocampus to predictive coding in visual cortex. Nat Neurosci 19:665-667 |
