Multisensory processing not only contributes to our complex perceptual fabric, but also underlies such disparate effects as speech comprehension as well as detection and orientation behaviors. Consequently, it would be expected that multisensory circuitry would be as varied as its behavioral/perceptual involvements. However, since 1966 (Horn and Hill) there has been only a single model of multisensory convergence and processing: the bimodal neuron. This neuron exhibits suprathreshold excitatory responses to stimuli from more than one modality (e.g., visual alone, somatosensory alone) as well as can produce an integrated response when those stimuli are combined (e.g., visual and somatosensory). Although numerous studies have affirmed the role of bimodal integration (especially in the superior colliculus) in detection and orientation behaviors, it is unclear how these dramatic response changes might influence more subtle levels of activity underlying cortical perceptual functions. Toward that end, recent direct examinations of multisensory processing in cortex have identified a 'new' form of multisensory neuron (Meredith, 2002; Dehner et al., 2004; Allman and Meredith, 2007; reviewed by Driver and Noesselt, 2008). The subthreshold multisensory neuron is excited by stimulation from only one sensory modality, but that activity can be modulated (facilitated or suppressed) by concurrent stimulation in an otherwise ineffective modality. As a consequence, the range of multisensory architectures has expanded beyond that of the traditional bimodal neuron, and even suggests that multisensory convergence may occur along a continuum from bimodal integration at one end, through subthreshold modulation to non-integrative unimodal processing at the other. Because existing computer models of multisensory processing are inadequate to describe these new findings, and because computer models can explore different biological scenarios that are not experimentally accessible, the present series of experiments proposes a computational examination of the spectrum of multisensory processing. First, using 128- channel recording, real in vivo sensory and multisensory data will be obtained from networks of sensory cortical neurons, including unimodal as well as bimodal and subthreshold types. Second, that biological data will be used in a computer simulation that incorporates (a) an integrate and fire spiking neuron model, b) static and dynamic synaptic plasticity rules, (c) hierarchical network topology that includes both excitatory and inhibitory neurons, and (d) introduces randomness to the network connections. Each of these features is novel to models of multisensory processing. Furthermore, real neuronal response data to single-modality stimulation along with data obtained in response to combinations of the same stimuli (derived from in vivo experiments) will be used to design and validate the model. This simulation will be used not only to model the different forms of multisensory processing, but also examine their responses to different levels and sources of inhibition as well as test the numerous variations of the multisensory continuum hypothesis. Once the model is operational, a website will be constructed to allow anyone to submit their own data to test the model. In this way, multisensory data from additional neural areas and species (potentially from invertebrates to humans) will be collected and compared, thereby providing an international repository and clearing-house for multisensory processing. The intellectual merit of the proposed investigations includes not only the development of a complex and sophisticated model of multisensory neuronal and network processing using state-of-the-art computing theories and biological verification, but also will provide insight and testable hypotheses regarding the fundamental properties of multisensory processing itself and its relation to behavior, perception and perceptual disorders. The broad impact of these efforts will contribute to progress toward the ultimate challenge of building artificial systems that perform on par with humans and thus facilitate the development of effective artificial devices for applications in robotics and medicine. ? ? ?
