Real-world scenes comprise a blooming, buzzing confusion of information. Yet at any given moment, we can only perceive and understand a small portion of that information. What we see and understand is quite literally determined by where we look. But what determines where we look? This project seeks to answer this important question. The project investigates the idea that the meaning of a scene plays the key role in guiding our eyes. If our hypothesis is correct, then we should find that meaning predicts where people look. Such a result will advance scientific knowledge of how our brains and minds work. The results will also be useful to applied areas of computer science and engineering, contributing to increased US economic competitiveness. High-tech applications include virtual and augmented reality, artificial vision and gaze-based input systems, baggage screening, medical image assessment, satellite image analysis, and other computer-based vision systems. The project will also contribute to training of culture- and gender-diverse students and researchers in these high-tech and scientific fields, advancing the development of a diverse, globally competitive workforce.
Attentional guidance during scene viewing draws on both the visual properties of the scene and its semantic content. Although the role of visual properties (e.g., physical salience) on attentional guidance has been extensively studied, far less is known about how the semantic content of a scene guides attention. Recent work in my lab indicates that the influence of physical saliency is reduced or even eliminated when semantic content is available to guide attention. These findings have opened up important new questions about how and when meaning guides attention in scenes. The central idea behind the current work is to address these questions with a set of targeted experiments. The proposed experiments will integrate new methods for representing the meaning of local objects and scene regions with high-resolution eyetracking to measure the influence of knowledge on attentional guidance in real-world scenes. The research includes experiments motivated by four goals: (1) To compare the roles of contextualized versus context-free local scene meaning in attentional guidance. (2) To evaluate the causal relationship between local scene meaning and attentional guidance. (3) To test the generality of guidance by scene meaning across tasks and viewing time. (4) To compare the roles of physical versus semantic features in scenes using search target templates. The research will establish the foundation for a new theoretical approach to the representation of scene meaning and attentional guidance in scenes. The ultimate objective is to develop a theory of attentional guidance in scenes based on guidance by meaning.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
