This project concerns computational challenges in the integration of visual and auditory signals in the brain. The primate brain uses different coding formats to encode the locations of visual and auditory stimuli, with maps of visual space and meters or rate codes for auditory space. The disparity in coding format poses challenges for connecting the visual and auditory aspects of common underlying objects or events. The proposed experiments investigate the possibility that visual stimuli cause fundamental changes in auditory representations, by changing the overall structure of auditory spatial sensitivity (Aim 1) and/or by capturing temporal fluctuations in auditory- evoked activity (Aim 2). These experiments will shed light on the neural mechanisms that can support visual-auditory perceptual phenomena such as the use of lip reading cues to facilitate speech comprehension, and the use of visual information to help focus on a particular sound in a crowded auditory scene such as a cocktail party. These abilities are important in normal, aging, and sensory-impaired populations. The experiments involve a combined approach involving behavior, neurophysiology, and advanced statistical analyses concerning evaluation of fluctuating patterns of neural activity. The work is spearheaded by a collaborative team led by Profs. Surya Tokdar (Statistical Science, Duke University) and Jennifer Groh (Neurobiology, Duke University), and builds on analogies from technological systems such as the use of time division multiplexing to intersperse signals from different sounds into a fluctuating neural firing pattern. This computational approach will yield insights into how sensory systems overcome their differences to work in synergy with each other, particularly when there is competition among stimuli for a slot in neural representations.
Health Relevance This project investigates how the brain integrates visual and auditory information. Deficits in this ability contribute to poor communication performance in noisy scenes in central auditory processing disorder and age-related hearing loss. Insights from these experiments could ultimately lead to improvements in the treatment of these and related conditions.
