Much knowledge about how human brains process information and generate actions has been informed by carefully controlled experiments in laboratory settings. However, understanding the brain in action requires exploration of its functions outside structured tasks. The current project explores neural processing over many days using large-scale recordings of brain activity augmented with video, audio and depth camera recordings, all simultaneously and continuously monitoring a subject. Importantly, unlike the majority of existing studies, here the subjects receive no instructions but are simply behaving as they wish in their hospital room-including eating, sleeping, and conversing with family. The project will advance data-intensive science and human neuroscience, leveraging external monitoring of the subjects to interpret naturalistic neural activity. The results of this project will be catalytic in understanding of the human brain, opening the door to study of brain function outside the structured confines of laboratory experiments.
The neural decoding algorithms developed will be directly applicable to current Brain-Computer Interfacing (BCI) technologies, enabling the deployment of systems that can predict the user's needs and improve quality of life outside the laboratory. Further, ongoing collaborations with neurosurgeons focus on evaluating this novel data-intensive approach to ethological brain mapping and how it may complement existing clinical functional brain mapping. The project will support and enable the education of students at the intersection of data science and neuroscience, including training scientists at the undergraduate, graduate, and post-doctoral career stages. Results from the research will be distributed as open access publications and code repositories, supporting a commitment to reproducible science.
This proposal focuses on data-driven innovations to enable more accurate decoding and inference of actions from long-term, naturalistic neural recordings. The first aim proposes to develop algorithms for automated decoding of natural motor and speech behaviors. Unsupervised clustering will be used to discover coherent patterns in brain activity, and clusters will be annotated with behaviors automatically parsed from external monitoring streams. Motivated by the size of the dataset and substantial variety between individuals, this scalable computational approach circumvents tedious manual annotation and fine-tuning of parameters. The second aim proposes to infer networks of dynamic causality of cortical networks engaged in task-free, naturalistic behaviors. This aim focuses on testing the hypothesis that neural correlates of naturalistic behaviors differ from those of repeated, instructed behaviors. Functional networks and the dynamic causality of cortical areas will be explored using methods from nonlinear dynamical systems theory. These networks will be compared to results from clinical brain mapping. This project will improve state-of-the-art neural decoding in naturalistic contexts and uncover neural correlates of task-free behaviors in humans.
