The link between patterns of activity in the brain and human actions has been studied for a long time. Careful study of these patterns has led to great scientific and technological progress. Brain-computer interfaces and brain-controlled prosthetic limbs are two examples. These technologies depend strongly on the power to tie brain signals to (intended) actions. In this project, we use state-of-the-art methods in cognitive science to develop a precise mathematical model of the link between brain activity and human behavior. Our theory now needs to be tested with new experimental data in adverse conditions. The challenges that will test whether this model can predict the brain-behavior link in new experiments, new behaviors, and new measures of brain activity. A strong model of brain and behavior will improve our knowledge of the brain and help future research and technological development. It can also improve the accuracy of science or technology that uses brain signals, including brain-computer interfaces. The project will benefit researchers outside our lab in other ways. We will document analyses and experiments as part of a series of video lectures. We will also freely share our data, code, and methods online. This will help other researchers to verify our findings and to educate members of the general public who have an interest in cognitive neuroscience. Finally, the project will involve junior scientists who will receive training and start a career in neuroscience or cognitive science.
The primary advantage of joint modeling is that it improves researchers' ability to estimate parameters of neural, cognitive, or behavioral models by using constraints imposed by one or more additional data modes. This has already allowed us to construct genuine neurocognitive models that are jointly informed by behavioral and neural data. We have developed a multimodal sequential accumulation model that makes predictions about the combination of reaction time, accuracy, and neuroelectric data, and that allows for conclusions not possible from either type of data individually. We will now test the generalizability of this model to other contexts. After first training a model on a relatively small data set, we will extrapolate its predictions to (a) new tasks by the same participants; (b) new participants in the same task; (c) new paradigms (i.e., tasks with new response modalities) by the same participants; and (d) new tasks by new participants. For a rigorous test of the linkage between the neural and behavioral data, the experiments will involve manipulations that selectively affect the cognitive components (visual preprocessing, motor preparation time, evidence accumulation) as well as corresponding human behavior (reaction times, accuracies, and choice behavior) and electrophysiological signals (ERP latencies and magnitudes and EMG muscle preparation signatures).
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
