In real world situations, objects and event comprise multiple stimulus attributes. They emit or reflect light, make sounds, and have characteristic mass and tactile properties. While the nervous system initially processes each sensory modality independently by transducing a restricted portion of energy by specialized receptors, this information is ultimately combined at higher stations of the nervous system to generate unified percepts. Even though this is a fundamental process of sensory systems, very little is understood how information from different sensory systems is combined, and how this integrations results in the perception of single objects and events. One way in which the processes of multi-sensory integration can be explored is to compare the responses of single neurons to the perceptions generated by combined sensory stimuli. By comparing the neuronal responses between the physical stimuli and the resulting perceptions, one can begin to elucidate the underlying neuronal mechanisms of that perception. One can further test these hypotheses by taking advantage of stimuli that generate faulty perceptions, i.e. illusions. The goals of this research project are to explore how single neurons in the cerebral cortex combine visual and auditory information. Specifically, we wish to determine if multi-sensory stimuli are processed in regions of the cerebral cortex that are traditionally considered to be 'unimodal', or if multi-sensory integration does not occur until the traditionally considered 'multi-modal'areas. We will use combined auditory and visual stimuli that generate perceptual illusions in both the temporal and spatial domain to explore where neurons respond best to the physical characteristics of the stimuli (such as in the sensory receptors) and where neurons respond best to the resulting perceptions that we all experience. By tracking these responses throughout the cortical hierarchy we will begin to understand how and where these integrative processes take place. These insights will provide a cornerstone for understanding complex perceptions and cognitive ability.
