Drugs of abuse operate in part by subverting the adaptive mechanisms (algorithms) that are normally used to value actions and make decisions. It is well known that many if not most drugs of abuse affect midbrain dopamine systems, and in recent years detailed computational models of dopaminergic function (called Reinforcement Learning or RL models) have advanced to the point that they are now used profitably to interpret functional imaging experiments on reward learning in human subjects. These same models can also account for important features of the addicted state. RL models predict the existence and behavioral impact of a range of learning signals including expectation errors (ongoing differences between expectations and actual outcomes) and fictive errors (ongoing differences between `what might have happened'and actual outcomes). Consequently, the connection of RL models to dopamine systems immediately recommends their use as quantitative probes of learning and decision-making in addicted populations. Despite the intimate connection between RL models, midbrain dopamine systems, and reward-guided choice, no model-driven imaging approaches have been used to probe any addicted populations. In this proposal, we seek to fill this gap, and will pursue a rigorous, model-based approach to reward-dependent learning signals, their generation, and their mathematical character in humans undergoing functional magnetic resonance imaging while they execute sequential choice tasks. This effort will be carried out in healthy human subjects and smokers, and we have developed a substantial body of preliminary data to support the specific goals of this project. We have chosen to apply a model-based approach to smokers because they use a legal drug, there is less prevalence of poly-substance abuse, smoking is generally thought to be a gateway drug for other drugs of abuse, and smokers represent a large health burden on society especially in their later years. By using RL models to guide the design, analysis, and interpretation of a range of reward-harvesting experiments, this proposal will yield new insights into the computational underpinnings of reward- dependent choice and its pathological hijacking by a common drug of abuse.
/ RELEVANCE Our understanding of the information distributed by midbrain dopamine systems has grown dramatically in recent years to the point that computer models of drug addiction are now usefully employed in the design and interpretation of reward-dependent learning experiments in humans. Drugs of abuse operate in part by subverting these learning mechanisms, which are normally used to value actions and make decisions. In this proposal, we plan to use computational model-based imaging studies to understand how a `gateway'drug (nicotine) perturbs experiential and `fictive'learning signals that guide human decision-making. We have chosen to use apply a model-based approach to smokers because they use a legal drug, there is less prevalence poly-substance abuse, smoking is generally thought to be a gateway drug for other drugs of abuse, and smokers represent a large health burden on society especially in their later years.
Showing the most recent 10 out of 62 publications
