The concept of surprise is central to sensory processing, adaptation and learning, attention, and decision making. Yet, no widely-accepted mathematical theory currently exists to quantitatively characterize surprise elicited by a stimulus or event, for observers that range from single neurons to complex natural or engineered systems. This project develops a formal Bayesian definition of surprise that is the only consistent formulation under minimal axiomatic assumptions. Surprise quantifies how data affects natural or artificial observers, by measuring the difference between posterior and prior beliefs of the observers. Preliminary human eye-tracking experiments demonstrated that participants gaze towards surprising image regions, significantly more than expected by chance, while watching complex videoclips including TV and video games. What are the underlying neural mechanisms responsible for this behavior? This cross-disciplinary proposal addresses this question as follows:
- Theory and modeling, to investigate how surprise relates to previous notions of saliency and novelty, and may advantageously complement Shannon information when analyzing neural function and behavior.
- Monkey electrophysiology with simple surprising stimuli, to investigate how single-neurons along the sensorimotor processing stream (primary visual cortex, frontal eye fields, superior colliculus) may be modulated by surprise.
- Parallel human/monkey psychophysics/electrophysiology, to investigate, in natural situations and with more complex stimuli including TV programs, whether stimuli which attract human and monkey attention may carry more Bayesian surprise.
The theory of surprise developed here is applicable across different modalities, datatypes, tasks, and abstraction levels. It has potential for impacting science and engineering, and especially education in computer science, mathematics, information theory, statistics, psychology and biology. The research involves undergraduate, graduate, and postdoctoral trainees in a collaboration between a theory lab (Baldi), a modeling and psychophysics lab (Itti), and a monkey electrophysiology lab (Munoz).
