Multimodal non-invasive functional brain imaging has made a tremendous impact in improving our understanding of the neural correlates of human behavior, and is now an indispensable tool for systems and cognitive neuroscientists. We propose to develop state-of-the-art multimodal functional imaging fusion algorithms for accurate visualization of the brain's dynamic activity and high spatial and temporal resolution. We propose to develop algorithms that combine complementary high spatial resolution of functional MRI (fMRI) and high-temporal resolution of magnetoencephalography (MEG) and electroencephalography (EEG) data for high-fidelity reconstruction of brain activity. In recent years, our research group has developed a suite of novel and powerful algorithms for MEG/EEG imaging superior to existing benchmark algorithms, and we have compared these results with electrocorticography (ECOG). Specifically, our algorithms can solve for many brain sources, including sources located far from the sensors, in the presence of large interference from unrelated brain sources using fast and robust probabilistic inference techniques. Here, we propose to extend this success in M/EEG inverse algorithms into the domain of multimodal imaging data fusion. Our overall goal here is to ultimately produce robust, high fidelity videos of event-related brain activation at a sub-millimeter and sub-millisecond resolution from noisy MEG/EEG and fMRI data using state-of-the-art machine learning algorithms. Specifically, we propose to extend a powerful new algorithm that we have recently developed, called Champagne, into two new fusion algorithms that combine fMRI, MEG and EEG data in different ways. Performance of both algorithms will first be rigorously evaluated in simulations, including performance comparisons with existing benchmark fusion algorithms. Algorithms will then tested for consistency on four fMRI-MEG+EEG datasets from healthy controls obtained for identical paradigms (auditory, motor, picture naming and verb-generation) and two fMRI-EEG datasets (face and motion perception). Additional validation studies will also be performed on fMRI-MEG/EEG datasets obtained from epilepsy patients and compared to electrocorticography (ECoG). Following successful testing and evaluation, all algorithms developed in this grant proposal, as well as example validation datasets, will be distributed using NUTMEG (, an open-source software toolbox that we have developed.

Public Health Relevance

Multimodal non-invasive functional brain imaging has made a tremendous impact in improving our understanding of the neural correlates of human behavior, and is now an indispensable tool for systems and cognitive neuroscientists. With the development of appropriate analytical tools, multimodal functional brain imaging is in the process of revolutionizing the diagnosis and treatment of a variety of neurological and psychiatric disorders such as autism, schizophrenia, dementia, and epilepsy that affect tens of millions of Americans.

National Institute of Health (NIH)
National Institute of Neurological Disorders and Stroke (NINDS)
Exploratory/Developmental Grants (R21)
Project #
Application #
Study Section
Special Emphasis Panel (ZRG1-NT-L (09))
Program Officer
Babcock, Debra J
Project Start
Project End
Budget Start
Budget End
Support Year
Fiscal Year
Total Cost
Indirect Cost
University of California San Francisco
Schools of Medicine
San Francisco
United States
Zip Code
Ranasinghe, Kamalini G; Hinkley, Leighton B; Beagle, Alexander J et al. (2014) Regional functional connectivity predicts distinct cognitive impairments in Alzheimer's disease spectrum. Neuroimage Clin 5:385-95
Niziolek, Caroline A; Nagarajan, Srikantan S; Houde, John F (2013) What does motor efference copy represent? Evidence from speech production. J Neurosci 33:16110-6
Findlay, Anne M; Ambrose, Josiah B; Cahn-Weiner, Deborah A et al. (2012) Dynamics of hemispheric dominance for language assessed by magnetoencephalographic imaging. Ann Neurol 71:668-86
Owen, Julia P; Wipf, David P; Attias, Hagai T et al. (2012) Performance evaluation of the Champagne source reconstruction algorithm on simulated and real M/EEG data. Neuroimage 60:305-23
Dolberg, Rebecca; Hinkley, Leighton B N; Honma, Susanne et al. (2011) Amplitude and timing of somatosensory cortex activity in task-specific focal hand dystonia. Clin Neurophysiol 122:2441-51
Hinkley, Leighton B N; Vinogradov, Sophia; Guggisberg, Adrian G et al. (2011) Clinical symptoms and alpha band resting-state functional connectivity imaging in patients with schizophrenia: implications for novel approaches to treatment. Biol Psychiatry 70:1134-42