This project concerns computational challenges in the integration of visual and auditory signals in the brain. The primate brain uses different coding formats to encode the locations of visual and auditory stimuli, with maps of visual space and meters or rate codes for auditory space. The disparity in coding format poses challenges for connecting the visual and auditory aspects of common underlying objects or events. The proposed experiments investigate the possibility that visual stimuli cause fundamental changes in auditory representations, by changing the overall structure of auditory spatial sensitivity (Aim 1) and/or by capturing temporal fluctuations in auditory- evoked activity (Aim 2). These experiments will shed light on the neural mechanisms that can support visual-auditory perceptual phenomena such as the use of lip reading cues to facilitate speech comprehension, and the use of visual information to help focus on a particular sound in a crowded auditory scene such as a cocktail party. These abilities are important in normal, aging, and sensory-impaired populations. The experiments involve a combined approach involving behavior, neurophysiology, and advanced statistical analyses concerning evaluation of fluctuating patterns of neural activity. The work is spearheaded by a collaborative team led by Profs. Surya Tokdar (Statistical Science, Duke University) and Jennifer Groh (Neurobiology, Duke University), and builds on analogies from technological systems such as the use of time division multiplexing to intersperse signals from different sounds into a fluctuating neural firing pattern. This computational approach will yield insights into how sensory systems overcome their differences to work in synergy with each other, particularly when there is competition among stimuli for a slot in neural representations.

Public Health Relevance

Health Relevance This project investigates how the brain integrates visual and auditory information. Deficits in this ability contribute to poor communication performance in noisy scenes in central auditory processing disorder and age-related hearing loss. Insights from these experiments could ultimately lead to improvements in the treatment of these and related conditions.

Agency
National Institute of Health (NIH)
Institute
National Institute on Deafness and Other Communication Disorders (NIDCD)
Type
Research Project (R01)
Project #
5R01DC016363-02
Application #
9507808
Study Section
Mechanisms of Sensory, Perceptual, and Cognitive Processes Study Section (SPC)
Program Officer
Poremba, Amy
Project Start
2017-07-01
Project End
2022-06-30
Budget Start
2018-07-01
Budget End
2019-06-30
Support Year
2
Fiscal Year
2018
Total Cost
Indirect Cost
Name
Duke University
Department
Psychology
Type
Schools of Arts and Sciences
DUNS #
044387793
City
Durham
State
NC
Country
United States
Zip Code
27705