The visual cortex is known to change its functional map and connectivity with other cortical regions following partial or full vision loss, including signi?cant repurposing among the other senses such as audition. What is unclear is whether such crossmodal incursion alters the multimodal pathways and multisensory integration of the various senses. Also, it is not yet known under what vision loss conditions (such as central vision loss with Age-Related Macular Degeneration, peripheral vision loss with Retinitis Pigmentosa, or full vision loss due to numerous causes) multisensory integration is facilitated (or suppressed), and whether these changes vary with retinal location. Such knowledge is critical to develop a more complete theoretical model of multisensory integration; to better evaluate potential for rehabilitation in those with vision loss; to provide a solid basis for the development of advanced retinal prostheses, sensory aids, and sensory substitution devices; and to develop optimal multisensory training and rehabilitation paradigms following visual restoration. To this end, we propose to determine the spatial and temporal characteristics of auditory-visual (A-V) integration in individuals with low vision (Speci?c Aim 1). More speci?cally, we will use a set of auditory-visual illusions as a psychophysical tool to determine the degree of A-V integration in various retinal locations as a function of both eccentricity from the fovea and proximity to regions of visual loss. We also propose to examine the viability of visual processing and crossmodal integration in those with low vision and the late blind by employing both A-V illusions and mental imagery (Speci?c Aim 2). We will determine whether multisensory integration from imagined visual stimuli can integrate with real auditory stimuli in the late blind to change the perceived location of auditory stimuli, including auditory spatial perception in the horizontal plane and in depth. The results from these two aims will provide an assessment of the key characteristics of auditory-visual interactions in the blind and those with low vision, and will identify differences in these multisensory interactions that are speci?c to the cause of vision loss. We also plan to identify the neural correlates of such crossmodal interactions and integrations using fMRI imaging (Speci?c Aim 3). We will examine key differences in visual cortical activity, as well as the connectivity patterns among auditory, visual, and multisensory cortices, by comparing low vision and late blind participants with sighted controls. A comprehensive understanding of the multisensory processing capabilities of low vision and late blind individuals will provide crucial insights into the consequences of functional reorganization in the human brain, and will also pave the way for advanced multisensory aids, visual prostheses, and rehabilitation protocols.
Although multisensory integration of sight and sound is well understood in sighted individuals, the extent to which the associated psychophysical and neural mechanisms of multisensory integration are altered following partial or complete vision loss remains unclear. The goal of this project is to develop a comprehensive understanding of the changes in multisensory integration of audition and vision that result from degenerative conditions such as Age-Related Macular Degeneration (AMD), Retinitis Pigmentosa (RP), and Non-Arteritic Anterior Ischemic Optic Neuropathy (NAION). The proposed studies will provide crucial insights into key characteristics of functional reorganization in the human brain, and will also pave the way for the next generation of multisensory aids, multisensory visual prostheses, and rehabilitation approaches.
