The visual input to the brain is transformed even before signals leave the eye, and these computations produce an efficient representation of the structure of the natural visual world. Previous work by the PI has shown that this processing can include repackaging of information for optimal prediction. This suggests a new approach to neural encoding. While many previous studies have sought to characterize what stimuli in the past gave rise to a subsequent response, this work asks what future stimuli those responses predict. The proposed project will derive the best possible predictor given the way objects move in the outside world and quantify how close the brain gets to this optimum. Viewing the brain through the lens of prediction develops a principle of neural coding and computation that can bridge brain regions, from the retina to higher visual areas. A component of this plan involves measuring and quantifying the predictive components of natural motion. In doing so, a public database of natural motion will be created that will be a lasting tool for the neuroscience and computer vision communities. An associated educational program will bring over 100 local middle school children to campus each year for hands-on neuroscience experiments, and will instill in a large group of graduate students the rewards and responsibilities of science teaching.
The research proposed here explores prediction in the visual system in a variety of ways: by computing efficiency bounds on the predictive encoding of complex motion, by developing quantitative methods to test these bounds in neural datasets, by measuring the statistics of motion in natural scenes, and by describing how, mechanistically, the brain achieves this performance. Hypotheses about how the brain performs optimal predictive computations may be constrained by the structure of predictable events in the natural visual world. To measure these statistics, a new natural movie database will be constructed by making high-speed, high-pixel-depth recordings of natural scenes. By quantifying motion in these data, this project will yield statistical and generative models of natural motion that will inform our understanding of the natural world and provide a compact way to recapitulate natural motion in silico. These stimuli will be used to test whether neural systems optimally encode information relevant for prediction. The work will also test what adaptive and otherwise non-linear processing steps underlie optimal prediction in the brain.
