This award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5).
Despite nearly half a century of research, the human ability to recognize objects visually remains a largely unsolved puzzle. Previous research on object recognition has primarily considered cases in which the target is viewed in isolation. However, the visual system can use contextual information -- such as the presence of other objects in the scene or knowledge about the kind of environment in which the object is found -- to determine the identity of an object as well. The contribution of this kind of information is especially clear when the image of the object itself is insufficient on its own. For example, a small yellow patch might be identified as a partially obscured banana in the context of a fruit bowl or as a leaf in the context of a tree. In an NSF-funded research project, Dr. Elan Barenholtz at Florida Atlantic University will use behavioral and computational techniques to examine two central questions regarding the role of context in object recognition: 1) How do people acquire knowledge about the relations between objects and their contextual scenes (for example, the likelihood of specific objects appearing in a certain type of context)? 2) How is this knowledge put to work in recognizing objects whose images have been degraded and cannot be recognized on their own? This research will employ two experimental methodologies: The first will use computer-generated artificial scenes in which participants must first learn the object/context relations from scratch and later use this knowledge in a recognition task. The second technique will test object recognition abilities in photographs of real world environments, including pictures of participants' own homes or workplaces. In this case, subjects will have knowledge about the expected object/context relations, based on their long-term experience, particularly when the environment is highly familiar to them. Human performance in these tasks will be assessed using statistical methods to assess the contribution of contextual information in object recognition.
Understanding human visual object recognition holds great promise for brain science -- as much as a third of the human cortex is thought to be devoted to visual processing. Such understanding is also important for designing artificial vision systems, which carry an enormous array of potential applications. However, previous theoretical techniques, which focused on specialized algorithms for extracting 3-dimensional structure from individual objects, have proven largely unsuccessful. Dr. Barenholtz's research represents a strong departure from earlier approaches, as it assumes that visual recognition relies on inferential strategies that draw on an individual's broad knowledge about the world and his or her experience with specific environments. This approach treats vision as relying on similar tools as other cognitive processes, such as inference and decision-making, suggesting that there may be a great deal of previously unexplored common ground across these different disciplines. By putting the "cognition" back into "recognition," this research has the potential to contribute to some long awaited breakthroughs in the field of visual recognition.
Although people are able to visually recognize objects effortlessly, our understanding of the underlying cognitive processes supporting this ability remains poor. Previous approaches to visual object recognition have focused on objects viewed in isolation. However, objects typically appear within a rich context, which may carry a significant amount of information about the object’s identity. For example, a small yellow patch visible in a fruit bowl has a high probability of being a banana. The brain may leverage this information when attempting to recognize an object. In the real world The current research investigated this potential role of context in facilitating recognition using two experimental paradigms. First, we examined how much information is available from contexts in natural scenes, using photographs of actual indoor living environments. Objects within these scenes were selected as targets and then their images were degraded by sequentially reducing their spatial resolution. We then tested to see at what level of resolution people were able to accurately identify the object when it was shown in its original context vs. when it was shown by itself. We also tested performance when the photographed scene from which the target was derived was from the participant’s own home. And was thus highly familiar. Overall, we found that viewing and object in context greatly facilitated recognition performance, allowing participants to identify the object at extremely low resolutions when the scene was familiar, at intermediate levels or resolution when the scene was unfamiliar and only at high levels of resolution when the object was shown without context. The facilitative effects of an unfamiliar context were particularly pronounced when the objects were independently rated as having a high degree of ‘fit’ within their contextual scenes (e.g. a toaster in a kitchen) vs. when they were rated as having a lower degree of fit (e.g. a toaster in a bedroom). These results suggest that people use their personal memories of environments with which they have experience, as well as their more general knowledge of what objects tend to appear within certain types of scenes., in order to identify objects A second study of natural scenes was conducted to determine what aspects of the context provide facilitative effects. We found that different types of contextual information yield differential effects depending on whether the context is familiar or not. For example, simply knowing the scene category (e.g. kitchen or living room) greatly enhanced performance when the scene was familiar but the specific location within the scene (e.g. on a table) was needed to produce enhanced performance when the scene was unfamiliar. Overall the results of these two studies suggest that context may play an important role in visual object recognition, particularly when the environment is familiar to the viewer, as is often the case under natural conditions. In the studies described above, participants came into the experiment with preexisting knowledge about the statistical relationships between objects and their contexts, limiting our ability to experimentally manipulate these factors. Thus we also employed a different experimental paradigm in which we generated artificial 3-D scenes in which we placed novel objects that participants had never encountered before. Using this method, we tested to see whether participants would quickly learn the locations of these novel objects within scenes (for example a certain object may always appear on a specific desk) and then use that information in order to recognize the objects when their images were degraded by blurring. We found that participants treated the location of an object as having equal importance to an ‘intrinsic’ feature of the object —in this case the object’s color—when trying to identify it. These results suggest that an object’s location within its context may be considered a ‘feature’ on par with other, previously established properties such as an object’s shape, texture, size, etc.. Overall, these results suggest the importance of considering context in theories of recognition and human visual performance.
