Methods known as 'multivariate pattern' (MVP) analysis can be used to decode the information patterns in brain activity obtained using functional magnetic resonance imaging (fMRI). However, a new decoding model has to be built for each brain, because two brains (and the representational spaces they employ) are difficult to align at a fine spatial scale. As a consequence, we do not yet know if different brains use the same codes or idiosyncratic codes to represent the same things. With funding from the National Science Foundation, Drs. James V. Haxby of Dartmouth College, and Peter J. Ramadge of Princeton University, in collaboration with Michael Hanke of the University of Magdeburg (Germany), are developing new methods to discover a coding scheme that works accurately across different brains. The methods being developed align brain activity across brains by projecting individual brain data into a common, high-dimensional space. This approach allows the researchers to build models of brain representational spaces for different cortical areas that are valid both across brains and across a wide range of stimuli and cognitive states. The researchers are developing two algorithms. One is referred to as 'hyperalignment' and the other as 'functional connectivity hyperalignment.' Hyperalignment rotates the voxel spaces (i.e., the smallest units in a brain image) of individual brains into a single high-dimensional space, in which each dimension is a profile of differential responses to stimuli, that is common across brains. Functional connectivity hyperalignment aligns voxel spaces based on the functional connectivity profile (i.e., relationships among activated brain areas) for each cortical location. Functional connectivity profiles allow for models of areas that do not respond to external stimuli in a consistent manner, for example, those areas in the so-called 'default-intrinsic system' that plays a central role in social cognition. The investigators are an interdisciplinary partnership - cognitive neuroscientists and signal-processing engineers - who have been working together successfully for several years.
Developing the computational methods to build common models of representational spaces will augment the power of brain activity decoding techniques, making it possible to investigate how finer, more detailed information is embedded in brain activity patterns, and to read out that information from functional brain imaging data. The proposed methods also will allow extension of brain decoding to the neural codes that underlie social cognition, that is, the representation of knowledge about the personal traits and mental states of others. These models also will allow investigation of how neural coding is altered within brain regions that are affected by experience, by development, and by psychopathology.
This project is jointly funded by Collaborative Research in Computational Neuroscience and the Office of International Science and Engineering. A companion project is being funded by the German Ministry of Education and Research (BMBF).
This grant supported research on the development of new representations of multi-subject fMRI data aimed at identifying and representing the shared cognitive response elicited under a common rich visual and auditory stimulus. Since every healthy subject’s brain is both anatomically and functionally slightly different, in order to make best use of multi-subject fMRI data, we need techniques that go beyond simply aligning anatomy, and spatial smoothing. The research supported under this grant took up this challenge. The objective is to identify a common representational space for the all subject’s data, and within this space to identify a component that is shared across subjects. The shared component can be regarded as the shared response elicited by the stimulus. Our approach collects the data from subject j into a matrix X(j). For example, this might be data from an anatomically selected region of interest (e.g., temporal cortex), or from a searchlight of voxels. We then seek a subject specific transformation R(j) of each subject’s data into a common (or shared) space: T = X(j)R(j) + E(j). Here E(j) is the error in the alignment of the data across subjects. In the common representational space T, we can compare each subject’s data, denoise the data, and make predictions of one subject’s response based on the aggregate response of the other subjects. One important problem is to identify the dimension of this common representational space. For example, data may be collected over time from tens of thousands of voxels (volumetric units of space), but the number of underlying degrees of freedom in the data may be far less than the number of voxels. So, roughly, one seeks a common representation space of the smallest dimension that captures the shared response across subjects in some meaningful way. The primary thrust of this research is directed to achieve inter-subject alignment of functional brain imaging data with a mathematical method that we call hyperalignment. We have completed the developed of a functional response version of the hyperalignment algorithm. This is based on subject specific orthogonal transformations of the fMRI data into a common aligned space. The basic approach, and an initial evaluation of its merits, appeared in a 2011 journal article in Neuron. We subsequently developed a regularized version of the functional response hyperalignment algorithm. This makes interesting connections with canonical correlation analysis. Regularization of hyperalignment can have significant benefits for small datasets. We believe this is important since most fMRI datasets fall into this category. This work was published as archival article in the Proceedings of the 2012 IEEE Statistical Signal Processing Workshop. We also investigated a kernelized version of functional response hyperalignment. This allows for a more general nonlinear alignment process. It is also more efficient. A paper reporting the method and results of this development was presented at the Neural Information Processing Systems Conference (NIPS) in December, 2012. The paper was selected for a spotlight presentation at the conference. We then used our method of aligning data in a common space to investigate to accomplish "collaborative denoising" of multi-subject fMRI data using the common features of the hyperaligned multi-subject data. The idea is that in the aligned space the data can be filtered across subjects to reduce noise, then the de-noised data can be projected back into the original voxel spaces. This work was presented as a spotlight talk and poster at New York Academy of Sciences 7th Annual Symposium of Machine Learning, October, 2012. The presentation received one of the best presentation prizes. The work subsequently appeared as an archival paper in the IEEE flagship conference on signal processing (ICASSP) in 2013. A second research thrust proposes to achieve inter-subject hyperalignment by hyperaligning intra-subject patterns of cortical functional connectivity. An initial investigation of an extension of functional response hyperalignment to functional connectivity alignment has revealed some new problems to be solved. The connectivity version of hyperalignment is a quadratic problem and does not have a unique solution. In an effort to bypass this issue we began a deeper investigation of combining functional response hyperalignment and connectivity hyperalignment. This required us to first examine connectivity alignment in an anatomical framework. The results of this investigation are presented in our journal paper "Inter-subject alignment of human cortical anatomy using functional connectivity", published in NeuroImage in 2013. We have initiated an investigation of how we can combine the idea of hyperalignment to include simultaneous dimensionality reduction. This has required us to rework the hyperalignment framework into a generative model. Initial results of this approach appear promising: it can achieve slightly improved performance while only selecting around 50 shared features, and is faster to compute than the original hyperaligment method. This version of the algorithm is still under investigation and needs further development before it matures. Nevertheless, it appears to be very promising.
