The study of the psychological, computational and neurobiological basis of Pavlovian conditioning is one of the longest standing research questions in psychology and neuroscience. In spite of the ubiquity and the importance of this form of learning, the computational mechanisms underlying the learning and expression of Pavlovian associations' remains poorly understood. Here, we investigate whether or not there exists two distinct forms of Pavlovian conditioning, a model-based form in which the expression of conditioned responses to a conditioned stimulus is sensitive to the incentive value of the associated unconditioned stimulus (US), and another model-free form in which conditioned responses elicited by a conditioned stimulus are insensitive to the current US value. The distinction between model-based and model-free reinforcement-learning mechanisms has received strong empirical support in the domain of instrumental conditioning, but little is known about whether or not a similar dichotomy exists in Pavlovian conditioning. Understanding the nature of the encoding of Pavlovian associations in the brain is important because of the critical role that learned Pavlovian associations might play in the maintenance of addiction, in which cues linked to drug outcomes might promote or invigorate responding for drugs, even if those drugs are no longer deemed valuable/desirable to the individual. In the present application we address this goal by performing both functional magnetic resonance imaging (fMRI) and single-unit recordings in humans while they undergo sequential Pavlovian conditioning with appetitive outcomes. We will use a number of different cutting-edge experimental and analytical techniques, including computational based analyses, multivariate pattern classification and high-resolution fMRI. We will test for the existence of these different representations in a number of distinct structures in the brain including the amygdala, orbitofrontal cortex, ventral striatum and dopaminergic midbrain. Because we will be using high- resolution fMRI, we will have the capacity to resolve the contribution of distinct sub-regions within these brain structures to model-based and model-free Pavlovian learning, including the basolateral versus centromedial amygdala, the human homologue of the core versus shell of the accumbens, different sectors of orbitofrontal cortex, and dorsal versus ventral parts of the substantial nigra and ventral tegmental area. To complement the fMRI studies, we will record from neurons primarily in the amygdala and orbitofrontal cortex in human neurosurgery patients while they while they perform one of the main tasks used in the fMRI studies, thereby enabling us to gain insight into the relationship between the observed fMRI signals and underlying neuronal activity in at least two of our key regions of interest. By combining across these different techniques and methodologies, we will be able to address the question of whether or not model-based and model-free forms of Pavlovian conditioning are implemented in parallel in the brain, and begin to gain insight into the specific contributions of different brain regions towards these two very distinct forms of learning.

Public Health Relevance

In spite of the ubiquity and importance of Pavlovian conditioning for both animal and human behavior, the computational underpinnings of this learning mechanism remain poorly understood. Here we test the hypothesis that there exist two distinct mechanisms for Pavlovian conditioning, a model-based form in which knowledge of the structure of the environment is used to flexibly compute Pavlovian values, and a model- free form in which the associative strength or value is based on the history of prior outcomes. Using both high resolution fMRI and single-unit neurophysiology in humans, in combination with novel computational and multivariate analysis techniques, we will address whether key brain regions involved in Pavlovian conditioning contribute to model-based learning or model-free learning of Pavlovian associations.

Agency
National Institute of Health (NIH)
Institute
National Institute on Drug Abuse (NIDA)
Type
Research Project (R01)
Project #
3R01DA040011-03S1
Application #
9671098
Study Section
Cognition and Perception Study Section (CP)
Program Officer
Grant, Steven J
Project Start
2016-07-01
Project End
2021-03-31
Budget Start
2018-04-01
Budget End
2019-03-31
Support Year
3
Fiscal Year
2018
Total Cost
Indirect Cost
Name
California Institute of Technology
Department
Type
Schools of Arts and Sciences
DUNS #
009584210
City
Pasadena
State
CA
Country
United States
Zip Code
91125