We are constantly faced with the trade-off between exploiting past actions with known outcomes and exploring novel actions whose outcomes may be better. When environmental rewards are stable, it is preferable to perform actions known to be rewarding, but when the environment is changeable, it is adaptive to explore alternatives and revisit actions whose value may have changed. This balance between exploitation and exploration is thought to rely on two interacting systems, namely modulation of corticostriatal circuits by dopaminergic neurons of the substantia nigra pars compacta (SNc), and modulation of anterior cingulate cortex (ACC) processing by noradrenergic neurons of the locus coeruleus (LC). However, little is known about the dynamics of these systems during exploitative and exploratory states, and even less is known about how activity in these systems impacts downstream circuits for action selection. Importantly, the decision processes underlying these behavioral states and the neural systems supporting them are central to the dysfunctions seen in a range of mental illnesses, such as Obsessive-compulsive disorder and Schizophrenia, among others. Here, we propose to explore the ways in which dopaminergic and noradrenergic transmission evolve during exploratory and exploitative behavioral states, and how these alterations impact coding in downstream circuits of the dorsal striatum (DS) and ACC. We begin by developing a behavioral paradigm to capture exploratory and exploitative action selection, using reinforcement learning models to quantify behavior. We will perform calcium imaging during this task to relate activity in dopaminergic neurons in SNc and noradrenergic neurons in LC to action selection, characterizing network states with advanced computational methods, as well as confirming the relationship between cell firing and downstream neuromodulator release with fast-scan cyclic voltammetry and microdialysis. We will then investigate how brief or sustained stimulation of dopaminergic and noradrenergic populations differentially impacts DS and ACC circuits, respectively, with particular focus on receptors tuned to brief (D1/?2) or sustained (D2/?1) changes in neuromodulator levels. This work will be conducted in the vibrant research community at Columbia University under the supervision of Drs. Rui Costa and Stefano Fusi. In additional to technical expertise, both Drs. Costa and Fusi have an impressive track record of successful trainees. The candidate has also assembled a team of expert collaborators, including Dr. Eleanor Simpson, Dr. Julia Sable, and Dr. Darcy Peterka. The entire mentoring team will guide the candidate in technical and professional training. The proposed work also has direct implications for a range of mental disorders, with the potential to identify novel therapeutic targets. Together, the proposed experiments will greatly clarify the role of dopaminergic and noradrenergic circuits in modulating behavioral variability, as well as the neural basis for our ability to balance the exploitation of known behaviors with the discovery of novel, creative responses to an ever-changing environment.
Individuals affected by any of a number of mental illnesses often experience difficulty choosing appropriate actions or suppressing impulses, and these deficits are thought to be due to abnormal functioning of dopaminergic and noradrenergic systems in the brain. However, our understanding of the role these systems play in choice remains limited. The proposed work will investigate signaling in these systems during action selection and how changes in dopamine and norepinephrine impact dynamics in downstream brain regions.
