Multimodal non-invasive functional brain imaging has made a tremendous impact in improving our understanding of the neural correlates of human behavior, and is now an indispensable tool for systems and cognitive neuroscientists. We propose to develop state-of-the-art multimodal functional imaging fusion algorithms for accurate visualization of the brain's dynamic activity and high spatial and temporal resolution. We propose to develop algorithms that combine complementary high spatial resolution of functional MRI (fMRI) and high-temporal resolution of magnetoencephalography (MEG) and electroencephalography (EEG) data for high-fidelity reconstruction of brain activity. In recent years, our research group has developed a suite of novel and powerful algorithms for MEG/EEG imaging superior to existing benchmark algorithms, and we have compared these results with electrocorticography (ECOG). Specifically, our algorithms can solve for many brain sources, including sources located far from the sensors, in the presence of large interference from unrelated brain sources using fast and robust probabilistic inference techniques. Here, we propose to extend this success in M/EEG inverse algorithms into the domain of multimodal imaging data fusion. Our overall goal here is to ultimately produce robust, high fidelity videos of event-related brain activation at a sub-millimeter and sub-millisecond resolution from noisy MEG/EEG and fMRI data using state-of-the-art machine learning algorithms. Specifically, we propose to extend a powerful new algorithm that we have recently developed, called Champagne, into two new fusion algorithms that combine fMRI, MEG and EEG data in different ways. Performance of both algorithms will first be rigorously evaluated in simulations, including performance comparisons with existing benchmark fusion algorithms. Algorithms will then tested for consistency on four fMRI-MEG+EEG datasets from healthy controls obtained for identical paradigms (auditory, motor, picture naming and verb-generation) and two fMRI-EEG datasets (face and motion perception). Additional validation studies will also be performed on fMRI-MEG/EEG datasets obtained from epilepsy patients and compared to electrocorticography (ECoG). Following successful testing and evaluation, all algorithms developed in this grant proposal, as well as example validation datasets, will be distributed using NUTMEG (, an open-source software toolbox that we have developed.

Public Health Relevance

Multimodal non-invasive functional brain imaging has made a tremendous impact in improving our understanding of the neural correlates of human behavior, and is now an indispensable tool for systems and cognitive neuroscientists. With the development of appropriate analytical tools, multimodal functional brain imaging is in the process of revolutionizing the diagnosis and treatment of a variety of neurological and psychiatric disorders such as autism, schizophrenia, dementia, and epilepsy that affect tens of millions of Americans.

National Institute of Health (NIH)
National Institute of Neurological Disorders and Stroke (NINDS)
Exploratory/Developmental Grants (R21)
Project #
Application #
Study Section
Special Emphasis Panel (ZRG1-NT-L (09))
Program Officer
Babcock, Debra J
Project Start
Project End
Budget Start
Budget End
Support Year
Fiscal Year
Total Cost
Indirect Cost
University of California San Francisco
Schools of Medicine
San Francisco
United States
Zip Code
Subramaniam, Karuna; Gill, Jeevit; Slattery, Patrick et al. (2016) Neural Mechanisms of Positive Mood Induced Modulation of Reality Monitoring. Front Hum Neurosci 10:581
Dale, Corby L; Brown, Ethan G; Fisher, Melissa et al. (2016) Auditory Cortical Plasticity Drives Training-Induced Cognitive Changes in Schizophrenia. Schizophr Bull 42:220-8
Hinkley, Leighton B N; Marco, Elysa J; Brown, Ethan G et al. (2016) The Contribution of the Corpus Callosum to Language Lateralization. J Neurosci 36:4522-33
Vossel, Keith A; Ranasinghe, Kamalini G; Beagle, Alexander J et al. (2016) Incidence and impact of subclinical epileptiform activity in Alzheimer's disease. Ann Neurol 80:858-870
Englot, Dario J; Hinkley, Leighton B; Kort, Naomi S et al. (2015) Global and regional functional connectivity maps of neural oscillations in focal epilepsy. Brain 138:2249-62
Ranasinghe, Kamalini G; Hinkley, Leighton B; Beagle, Alexander J et al. (2014) Regional functional connectivity predicts distinct cognitive impairments in Alzheimer's disease spectrum. Neuroimage Clin 5:385-95
Kort, Naomi S; Nagarajan, Srikantan S; Houde, John F (2014) A bilateral cortical network responds to pitch perturbations in speech feedback. Neuroimage 86:525-35
Sekihara, Kensuke; Nagarajan, Srikantan S (2013) Residual coherence and residual envelope correlation in MEG/EEG source-space connectivity analysis. Conf Proc IEEE Eng Med Biol Soc 2013:4414-7
Tarapore, Phiroz E; Findlay, Anne M; Lahue, Sara C et al. (2013) Resting state magnetoencephalography functional connectivity in traumatic brain injury. J Neurosurg 118:1306-16
Owen, Julia P; Li, Yi-Ou; Ziv, Etay et al. (2013) The structural connectome of the human brain in agenesis of the corpus callosum. Neuroimage 70:340-55

Showing the most recent 10 out of 20 publications