Visual information is initially received by the eyes; however, the experience of vision itself is the result of complex computations carried out deep within the brain. Though many aspects of human vision are well-understood, the mechanisms by which the visual system maps recent memory (what happened a moment ago) onto incoming visual signals (what is happening now) remain a critical question for cognitive science. As events unfold in real time, there needs to be a mechanism for continually linking the current visual information with what just happened. For example, tracking a moving object requires mapping very recent visual memory signals onto the visual signals currently entering from the retina and tying this information together. The same holds true when trying to detect changes in a scene, or when trying to remember the positions of groups of objects, or even when trying to link landmarks on a map to landmarks in real space. This research project aims to characterize the correspondence mechanisms that enable this mapping between the recent and the current. Understanding how the human visual system accomplishes correspondence mappings is critical for understanding complex visual activities and the development of specialized visual skills such those involved in driving, radar control, video analysis, satallite imagery analysis, and baggage screening. Understanding correspondence mechanisms could also inform the design of artificial vision systems.
The research achieves its goals by combining eye tracking methodology and probabilistic models derived from computer vision algorithms, along with behavioral tasks that engage motion tracking, spatial working memory, and visual working memory. Eye tracking is a crucial component of the project because, in humans, the quality of received visual signals depends heavily on a source's distance from an observer's fixation (eccentricity). The research project therefore begins with experiments that compare observers' performance as a function of their fixations as well as simulations by computational models that adopt those empirically obtained fixations. Subsequent experiments then investigate methods for facilitating and training fixation to improve observer performance. A major implication of the research is that fixation selection places tremendous constraints on the accuracy of probabilistic correspondence algorithms, and as a result, on the ability to effectively obtain, store, and retrieve visual information.
