Reward-predictive cues play a key role in substance use disorders (SUDs) by driving craving and reward-seeking behaviors despite adverse consequences, making it essential to understand how predictive credit is assigned. The current consensus is that cue-reward learning depends on the computation of prediction error (PEs) by the dopaminergic system?the teaching signal that drives the updating of associative representations in downstream areas. The question of how predictive credit is assigned may thus come down to how PE are computed. Early learning models assumed that each cue generates its own separate PE, implying that credit assignment is simply determined by each cue?s correlation with reward independently of other cues. Since the discovery of cue-competition phenomena, however, the dominant view has been that reward learning is driven by a single, aggregate PE shared among all cues, as a result of which only the best reward predictors will acquire substantial credit and outcompete less well correlated cues. Criticaly, converging evidence suggests that competitive and noncompetitive learning might be the two ends of a learning-style spectrum of individual differences, with deep implications for SUDs. We propose that these differences may constitute a risk factor for SUDs because the less competitive learning is, the more credit will be assigned to as host of cues that would otherwise get outcompeted. The overarching goal of this research project is to elucidate the neural mechanisms that determine an individual?s learning style. As a first step in this direction, this proposal focuses on the role of the orbitofrontal cortex, a region that has been implicated in various aspects of reward processing. We previously hypothesized that OFC acts as a major hub where reward expectancies elicited by multiple cues can be summed to form an aggregate expectancy?the key step to computing an aggregate PE. Thus, a clear prediction of this hypothesis is that OFC should play a critical role in competitive learning. The experiments in this proposal will test this prediction using in-vivo electrophysiological (Aims 1 and 2) and optogenetic stimulation (Aim 3) in OFC in combination with a novel task specifically designed to examine individual differences in learning style and their neural substrates. This task is unique in that it sets competitive and noncompetitive learning styles in conflict with one another. Preliminary results using this task has confirmed individual differences that correlate with the extent to which neural activity in OFC encodes reward expectancies based on competitive vs. noncompetitive styles. Interestingly, both styles are encoded during learning but show different temporal dynamics within the cue epoch, suggesting the interplay between two relatively independent learning systems. In addition, our pilot data indicates a causal role of OFC in competitive learning. Findings from this proposal will thus further our understanding of reward learning mechanism, which is essential for developing new therapeutic strategies for SUDs.
The notion that cue-reward learning is a competitive process governed by an aggregate prediction error has dominated learning-theory and reinforcement-learning models as well as neuroscience research. This proposal will test the hypotheses that competitive and noncompetitive learning are two ends of a continuum of individual differences, and that the orbitofrontal cortex plays a fundamental role in the former. The results will provide key insights into the neural mechanisms that determine an individual?s learning style and may contribute to his/her vulnerability to SUDs.
