Drugs of abuse operate in part by subverting the adaptive mechanisms (algorithms) that are normally used to value actions and make decisions. It is well known that many if not most drugs of abuse affect midbrain dopamine systems, and in recent years detailed computational models of dopaminergic function (called Reinforcement Learning or RL models) have advanced to the point that they are now used profitably to interpret functional imaging experiments on reward learning in human subjects. These same models can also account for important features of the addicted state. RL models predict the existence and behavioral impact of a range of learning signals including expectation errors (ongoing differences between expectations and actual outcomes) and fictive errors (ongoing differences between `what might have happened'and actual outcomes). Consequently, the connection of RL models to dopamine systems immediately recommends their use as quantitative probes of learning and decision-making in addicted populations. Despite the intimate connection between RL models, midbrain dopamine systems, and reward-guided choice, no model-driven imaging approaches have been used to probe any addicted populations. In this proposal, we seek to fill this gap, and will pursue a rigorous, model-based approach to reward-dependent learning signals, their generation, and their mathematical character in humans undergoing functional magnetic resonance imaging while they execute sequential choice tasks. This effort will be carried out in healthy human subjects and smokers, and we have developed a substantial body of preliminary data to support the specific goals of this project. We have chosen to apply a model-based approach to smokers because they use a legal drug, there is less prevalence of poly-substance abuse, smoking is generally thought to be a gateway drug for other drugs of abuse, and smokers represent a large health burden on society especially in their later years. By using RL models to guide the design, analysis, and interpretation of a range of reward-harvesting experiments, this proposal will yield new insights into the computational underpinnings of reward- dependent choice and its pathological hijacking by a common drug of abuse.

Public Health Relevance

/ RELEVANCE Our understanding of the information distributed by midbrain dopamine systems has grown dramatically in recent years to the point that computer models of drug addiction are now usefully employed in the design and interpretation of reward-dependent learning experiments in humans. Drugs of abuse operate in part by subverting these learning mechanisms, which are normally used to value actions and make decisions. In this proposal, we plan to use computational model-based imaging studies to understand how a `gateway'drug (nicotine) perturbs experiential and `fictive'learning signals that guide human decision-making. We have chosen to use apply a model-based approach to smokers because they use a legal drug, there is less prevalence poly-substance abuse, smoking is generally thought to be a gateway drug for other drugs of abuse, and smokers represent a large health burden on society especially in their later years.

Agency
National Institute of Health (NIH)
Institute
National Institute on Drug Abuse (NIDA)
Type
Research Project (R01)
Project #
5R01DA011723-13
Application #
8437267
Study Section
Cognitive Neuroscience Study Section (COG)
Program Officer
Grant, Steven J
Project Start
1998-04-01
Project End
2014-02-28
Budget Start
2013-03-01
Budget End
2014-02-28
Support Year
13
Fiscal Year
2013
Total Cost
$328,041
Indirect Cost
$109,646
Name
Virginia Polytechnic Institute and State University
Department
Type
DUNS #
003137015
City
Blacksburg
State
VA
Country
United States
Zip Code
24061
Kirk, Ulrich; Gu, Xiaosi; Harvey, Ann H et al. (2014) Mindfulness training modulates value signals in ventromedial prefrontal cortex through input from insular cortex. Neuroimage 100:254-62
Xiang, Ting; Lohrenz, Terry; Montague, P Read (2013) Computational substrates of norms and their violations during social exchange. J Neurosci 33:1099-108a
Lohrenz, Terry; Bhatt, Meghana; Apple, Nathan et al. (2013) Keeping up with the Joneses: interpersonal prediction errors and the correlation of behavior in a tandem sequential choice task. PLoS Comput Biol 9:e1003275
Montague, P Read; Dolan, Raymond J; Friston, Karl J et al. (2012) Computational psychiatry. Trends Cogn Sci 16:72-80
Kishida, Kenneth T; Yang, Dongni; Quartz, Karen Hunter et al. (2012) Implicit signals in small group settings and their impact on the expression of cognitive capacity and associated brain responses. Philos Trans R Soc Lond B Biol Sci 367:704-16
Downar, Jonathan; Bhatt, Meghana; Montague, P Read (2011) Neural correlates of effective learning in experienced medical decision-makers. PLoS One 6:e27768
Kirk, Ulrich; Downar, Jonathan; Montague, P Read (2011) Interoception drives increased rational decision-making in meditators playing the ultimatum game. Front Neurosci 5:49
Kishida, Kenneth T; Sandberg, Stefan G; Lohrenz, Terry et al. (2011) Sub-second dopamine detection in human striatum. PLoS One 6:e23291
Harvey, Ann H; Kirk, Ulrich; Denfield, George H et al. (2010) Monetary favors and their influence on neural responses and revealed preference. J Neurosci 30:9597-602
Helekar, Santosh A; Shin, Jae C; Mattson, Brandi J et al. (2010) Functional brain network changes associated with maintenance of cognitive function in multiple sclerosis. Front Hum Neurosci 4:219

Showing the most recent 10 out of 36 publications