Computer vision algorithms examine images and make sense of what these images depict. Current computer vision algorithms are able to interpret images at the level of a typical middle school student for many image interpretation tasks. Recent advances in computer vision have led to rapid technological advances which are still unfolding but affect not only the technology industry, but education, national security and health care. However, these new algorithms are as yet poorly understood and do not describe how natural learners such as a typical middle school student learn to understand the visual world. This proposal draws together a team of cognitive psychologists, neuroscientists, and computer scientists to develop a new class of algorithms for computer vision inspired by the way people learn.
The key insight of this proposal is that human learners, unlike many leading computer vision techniques, make extensive use of the temporal structure of visual experience to extract structure. In the real world the image on the human retina is almost never static. Changes in eye position and movements of the head and body create a rich and complex temporal structure over a range of scales from hundreds of milliseconds up to days and weeks. This proposal a) develops databases of realistic and dynamically changing images in the real world and in immersive virtual reality environments, b) develops computational models for learning visual representations from temporally structured experiences and, c) examines the brain structures supporting representations integrating time and space across scales using fMRI. The algorithms pursued in this project are inspired by recent theoretical work in the neuroscience of scale-invariant memory. However, because the databases will be made publicly available, other researchers will be able to develop other algorithms that exploit temporal and spatial correlations. Taken together, these efforts are intended to catalyze a new generation of techniques for human-like machine learning algorithms with applications in computer vision.
