There is a fundamental gap in current Brain Machine Interface (BMI) technology, the lack of BMIs' abilities to work in real world situations, and to adapt with the user. We suggest the production of an autonomously adapting BMI. A key scientific aspect of our work is to further elucidate our preliminary findings on reward modulation of the primary motor cortex (M1). This finding indicates that we could generate our fully autonomous BMI using one implant in M1 with the use of reinforcement learning to update the BMI. Humans are currently being implanted in M1 with the same electrode arrays we are using, and thus we may be able to test our system in the short term. Our long-term biomedical engineering goal is to develop a fully integrated BMI that allows its user to recognize the system as self and make natural looking accurate movements. This will ultimately require a system that adapts with the user and provides sensory feedback. Our long-term neuroscience goals are to determine the basic properties of the primary sensorimotor cortices, such as the influence of reward on synaptic plasticity within these regions and sensorimotor adaptation and learning. The goals of this proposal are threefold, we will prove that supervised reinforcement learning (RL) leads to robust BMIs, that there is reward modulation in M1 and quantify how this modulation influences more traditional M1 representations, such as movement kinematics. Our Central Hypothesis is that reward modulates M1 neural activity and this modulation can be tapped into for the purpose of an autonomous BMI, which will learn to work with the user based on the users interpretation of the BMIs performance. This hypothesis is driven by our preliminary work showing that reinforcement-learning methods can produce good BMI control and that reward modulation has an influence on M1 activity. We have decoded reward expectation from M1 during reaching movements and passive observation, however, this was between reward and non-reward, and did not control for motivation explicitly as we now propose.
Our specific aims are 1) examine if reward modulation has an influence on the primary motor cortex's representation of movement and action observation; 2) examine if there is an influence of reward modulation on the primary motor cortex under BMI control; and 3) examine the use of supervised reinforcement learning with a neurally derived evaluative signal for BMI control. Our contributions will be significant, in our opinion, because they will lead to enhanced BMIs that learn to work with the user for improved overall performance while adding significantly to our knowledge on M1. This should lead to increased independence for the BMI's user, and ultimately a decrease in health care costs. Not to mention an increased quality of life. The proposed research is innovative, in our opinion, because it pushes to move BMIs from the lab to real world capabilities, able to deal with changing environments autonomously. Our proposed work will help push us to new horizons ushering in the age of teaming between humans and altruistic computational agents.

Public Health Relevance

of our work to public health is the production of a sys- tem that aims to help individuals that can't make reaching and grasping movements, such as due to amputation, or neurologic conditions. Our system, a brain machine inter- face, will record neural activity and translate this into desired movements of prosthetic limbs, computer cursors etc. Innovative to our work is the derivation of the individual's interpretation f the movement, such as good or bad, from their recorded neural activity alone and to use this information to continually adapt the system autonomously.

National Institute of Health (NIH)
National Institute of Neurological Disorders and Stroke (NINDS)
Research Project (R01)
Project #
Application #
Study Section
Special Emphasis Panel (BNVT)
Program Officer
Ludwig, Kip A
Project Start
Project End
Budget Start
Budget End
Support Year
Fiscal Year
Total Cost
Indirect Cost
Suny Downstate Medical Center
Schools of Medicine
United States
Zip Code
Zhao, Yao; Hessburg, John P; Asok Kumar, Jaganth Nivas et al. (2018) Paradigm Shift in Sensorimotor Control Research and Brain Machine Interface Control: The Influence of Context on Sensorimotor Representations. Front Neurosci 12:579
Gao, Peng Penny; Goodman, Jeffrey H; Sacktor, Todd Charlton et al. (2018) Persistent Increases of PKM? in Sensorimotor Cortex Maintain Procedural Long-Term Memory Storage. iScience 5:90-98
McNiel, David; Bataineh, Mohammad; Choi, John et al. (2016) Classifier Performance in Primary Somatosensory Cortex Towards Implementation of a Reinforcement Learning Based Brain Machine Interface. Proc South Biomed Eng Conf 2016:17-18
Bataineh, Mohammad; McNiel, David; Choi, John et al. (2016) Pilot Study for Grip Force Prediction Using Neural Signals from Different Brain Regions. Proc South Biomed Eng Conf 2016:19-20
McNiel, David B; Choi, John S; Hessburg, John P et al. (2016) Reward value is encoded in primary somatosensory cortex and can be decoded from neural activity during performance of a psychophysical task. Conf Proc IEEE Eng Med Biol Soc 2016:3064-3067
Marsh, Brandi T; Tarigoppula, Venkata S Aditya; Chen, Chen et al. (2015) Toward an autonomous brain machine interface: integrating sensorimotor reward modulation and reinforcement learning. J Neurosci 35:7374-87