In daily life, we frequently experience correlated sensations across our different sensory modalities. For example, as we climb up the stairs, we receive sensations to our auditory, visual, tactile, and vestibular systems that are all related to the experience of stair-climbing. These types of multisensory experiences are a key aspect of how we interact with, and learn about, the world around us. Through our experience with the world, our sensory abilities undergo refinements that allow us to optimize our performance in the tasks that we perform. These sensory refinements involve fine-tuning of processing in each of our sensory modalities, but equally importantly, in how we merge information across modalities. While much research has focused on how learning can take place within each individual sensory system, the learning of how information is combined across the senses has been largely neglected. The PIs will conduct a series of experiments in which they can track visual, auditory, and auditory-visual multisensory learning in parallel, and discriminate among different theories of multisensory processing and learning. Behavioral and neuroimaging methods will be combined to shed light on the roles that different brain areas, and the interactions between brain areas, play in the process of multisensory learning. Altogether these studies will provide fundamental insights into how our sensory systems work together and refine their interactions to best operate in the tasks that we perform.
This project will be the first systematic investigation of multisensory perceptual learning. It will also be the first study of changes in interaction between brain areas that may occur as a result of sensory learning. Altogether, this study promises to provide foundational knowledge regarding the brain mechanisms involved in multisensory learning as well as the mechanisms of learning in general. Understanding multisensory learning can contribute to the development of more effective strategies for learning. These strategies can be utilized to enhance learning for typically-developed children and adults, as well as to facilitate learning and communication for individuals with deprivation in one sense (e.g., individuals with low-vision or low-hearing, patients with cochlear implants or undergoing macular degeneration or cataract surgeries). They can also contribute to devising remedial programs for dyslexia, which appears to involve deficits in combining information across the senses.
