Humans experience the world via multiple sensory modalities -- we see, we hear, and we touch. Each sensory modality has unique strengths and weaknesses in its ability to represent the world around us. Our minds are able to flexibly recruit visual or auditory brain structures when their strengths correspond to the task at hand. In addition, our experience at any moment depends both on sensory input and on working memory and attention mechanisms in the brain. These mechanisms have very limited capacities, which in turn limit cognition. The overarching goal of this project is to examine the human brain mechanisms that support attention and working memory in vision, hearing, and touch. The research team will perform functional MRI experiments to study the brain activity of healthy adults while they perform demanding sensory working memory tasks. Preliminary studies by the research group suggest that there are extensive brain networks, extending into the frontal lobes of the cerebral cortex, that are specialized for each sensory modality, as well as a shared network that supports and unifies these three senses. The current research program will examine individual differences in working memory performance and brain network organization. It will also develop advanced computational models that can predict the functional organization of an individual's brain from their unique pattern or 'fingerprint' of brain connectivity. The project will facilitate other research efforts through the dissemination of new models and computational tools, and will recruit and train young scientists, including members of groups that are under-represented in STEM, in cognitive neuroscience research.
This proposal has 4 primary intellectual goals: (1) identify the fine-scale organization of tactile, visual, auditory, and modality-independent attention & working memory (WM) regions within human cerebral cortex; (2) reveal the specificity of coding of WM information across cortical regions for each modality; (3) detail the network organization of attention & WM circuits; and (4) test hypotheses about content-specific WM mechanisms and cross-modality WM coding. Individual subject fMRI analyses permit fine-scale observation of distinct functional regions. Drawing on subject-specific maps of cortical organization, the research group will re-examine the highly debated question of which brain structures support stimulus-specific working memory for each modality. It will investigate the specificity of sensory modality biased regions by examining functional networks in the resting-state within individual participants. The research team will leverage its findings to probe these networks in 1200 subjects from the Human Connectome Project dataset. The research group will also test and validate a machine-learning approach, Connectome Fingerprinting, for predicting the location of modality-specific working memory regions in individual brains from their unique functional connectome. Vision excels in coding spatial information, but codes timing less reliably; conversely, the auditory system performs high-fidelity temporal coding but coarse spatial coding. The research team has observed cross-modal recoding of WM information into cortical structures that prefer the 'appropriate modality' - auditory spatial WM recruits visual-biased regions and visual temporal WM recruits auditory-biased regions. To probe content-specific WM mechanisms, the researchers will examine interactions between sensory modality WM and space/time WM. The research group hypothesizes that visual and spatial WM stores are distinct, but that auditory and timing WM stores are shared. Collectively, these studies will elucidate the human brain networks and mechanisms that support sensory working memory.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
