The human eye sends information to the brain at an estimated rate of approximately 10 megabits per second, roughly the speed of an ethernet connection. Processing such a large bandwidth stream of visual information on behaviorally relevant time scales requires the brain to extract and represent information from visual signals efficiently, i.e. represent the most information for the least cost in time, hardware and energy. In essence, the brain needs to compress the visual stream in much the same way that software compresses the digital representation of a movie. This coding enhancement might arise because the brain has evolved coding strategies that specifically account for the fact that because of both object and eye movements, the visual input to the eye may be correlated in space and time. As a result, the visual signals to the brain from the eye and retina may be quite predictable. One of the primary questions in current sensory-motor systems research is to what extent the brain utilizes prediction to compensate for the fact that it takes a finite amount of time to process information even though the visual scene might change in the interim. This proposal focuses on neural representation of visual motion and gaze behavior for natural motion videos and uses a novel video game environment to simplify the analysis of gaze. The project will also create a publicly available database of natural gaze recordings, analyze the statistics of natural retinal image motion, characterize the representation of naturally correlated motion stimuli in cortical neurons, and to articulate the strategy underlying gaze control. This database will benefit neuroscience, computer vision, media design, and other fields.
The experimental approach combines cortical physiology in non-human primates with high-resolution eye movement recording in both humans and monkeys. The PI proposes to use high-resolution videos of natural moving scenes as visual stimuli while recording neural activity in motion-sensitive visual cortex. By carefully degrading the movies to make them increasingly less natural and measuring the impact on neural responses, the experiments will determine what features of the moving visual scene are represented most precisely. A second set of experiments will study the interactions between the visual scene and eye movements. The PI will develop an innovative Pong-like video game that actively engages the viewers and creates a common viewing purpose (scoring points) while simplifying the identification of the target of interest to aid analysis, thereby controlling the cognitive state of the viewer. The interdisciplinary nature of the work will provide training opportunities for undergraduate and graduate students crossing over from mathematics and physics to neurobiology, and for students with a biology background to gain skills in computational analysis.
