The goal of this research is to understand how we see what we see: how does the brain analyze the light falling on the retina of the eye to reveal a world full of objects, people and things? During the past year we have focused on both (i) perception and (ii) memory of complex visual stimuli, in particular real-world visual scenes, objects and people (NCT00001360). Perception: Real-world scenes are incredibly complex and heterogeneous, yet we are able to categorize them and identify objects and people within those scenes effortlessly. While prior studies have identified brain regions that appear to be specialized for processing faces, object and scenes, it remains unclear what the precise roles of these different regions are and what information they contain. We presented participants with 144 real-world scenes representing a wide range of object (e.g. bags, pets) and scene (e.g. beach, mountain) categories. First, we presented those scenes to participants while we measured their brain activity using functional magnetic resonance imaging (fMRI). Based on the patterns of responses observed in different brain regions, we determined how each region represents the scenes and how the structure of those representations differs across regions. Second, we asked participants to arrange the 144 images on a large computer screen according to their similarity, with similar scenes placed close together and very different scenes placed far apart. This gave us a measure of the conceptual understanding of the scenes. We then compared the brain and behavioral data to determine which brain regions underlie our conceptual understanding. Surprisingly, we found only a limited correspondence between the behavioral and brain data. These results suggest a complex relationship between localized responses in high-level visual cortex and behavioral similarity judgments each domain reflects different properties of the images, and responses in high-level visual cortex may correspond to intermediate stages of processing between basic visual features and the conceptual categories that dominate the behavioral response. These results provide important insights into the brain mechanisms supporting our understanding of objects and scenes. However, this study focused on a relatively small number of object and scene categories. We are now substantially extending this study by collecting behavioral, fMRI and magnetoencephalography (MEG) data for nearly two thousand categories of objects that span our everyday experience. Memory: To understand the nature of object and scene memory, we have focused on two specific issues: 1) Organization of the medial parietal cortex Human medial parietal cortex (MPC) has been implicated in multiple cognitive processes including memory recall, visual scene processing and navigation. Given such diverse recruitment of MPC across cognitive domains historically considered largely independent, it is not surprising that there has been no clear consensus with regard to its function and overarching organization. In a series of studies, we measured brain activations with fMRI while we presented participants with images of scenes and people and also asked them to recall specific events (e.g. shaving, going to a beach) as well as recall famous or personally familiar places and people. Our findings revealed two key insights. First, we found an apparent gradient of representation from more perceptually related to more memory related from posterior to anterior within MPC. Second, we found distinct subdivisions within MPC that were active differentially during recall of either people or places. Collectively, these results provide a new framework for understanding the functional role of MPC and its significance during different aspects of cognition. 2) Elucidating the content of scene memory When we recall a previously experienced event, what exactly are we remembering? Are our memories a precise, high-definition recording of that event, a low-resolution gist of that memory, or even just a verbal description of what we saw? Answering this question is an essential component of being able to tease apart the mechanisms of memory: what information is encoded and maintained, how memory decays over time, and what information is retrieved from these memories. In this study, we investigated the content of memory for real world scenes using a drawing task. Participants studied 30 scenes and, after 10 minutes, drew as many images in as much detail as possible from memory. The resulting memory-based drawings were scored by thousands of online observers, revealing numerous objects, few memory intrusions, and precise spatial information. Our findings show that not only is it possible to quantify the content of memory during free recall, but those memories contain detailed representations of our visual experiences. Elucidating how the brain enables us to recognize objects, scenes, faces and bodies provides important insights into the nature of our internal representations of the world around us. Understanding these representations is vital in trying to determine the underlying deficits in many mental health and neurological disorders.
Showing the most recent 10 out of 44 publications
