A long-standing problem in cognitive neuroscience is understanding how we can categorize a novel scene in about the same amount of time that it takes to blink one's eyes. Categorization aids both identifying objects and locating them in cluttered scenes, and thus allows for intelligent action in the world. How do we derive semantically meaningful categories from the raw image pixels? Currently, there is experimental support for multiple mechanisms supporting scene categorization, such as through recognizing the scene's objects or other visual features such as spatial layout, color, or texture. Crucially, substantial correlations exist between all of these proposed features. This make it difficult to disentangle their relative contributions to categorization. For example, if two scenes share an object, they will often also share the texture features associated with that object. In this work, the PI (Dr. Bruce C Hansen, Colgate University) and co-PI (Dr. Michelle R Greene, Bates College) seek to disentangle the contribution of such features, and also to determine when these features become available for use, and how they combine to support scene categorization. By understanding the temporal dynamics of the brain activity related to scene categorization, it will be possible to obtain critical insights into how people rapidly but flexibly extract information from the environment. This work forms a bridge across several disciplines including psychology, cognitive neuroscience, computer vision, and machine learning. As such, the project will engage undergraduate students in truly interdisciplinary training that is at the cutting edge of multiple fields.
This project will make use of high-density EEG combined with machine learning, computational modeling behavioral measures, and advanced neuromodulation to determine how and when the behaviorally relevant features support scene categorization. First, the work will link the encoding of these features to visual event related potentials (vERPs) and also to category information using multivariate classification techniques from machine learning. Taken together, these techniques will allow the PIs to determine the unique contributions of each feature to category-related brain activity over time. A hallmark of intelligent action is flexibility. Therefore, the project will also investigate the flexibility of feature use by manipulating the diagnosticity of information available to observers. These studies will provide insights regarding feature space usage as a function of task demands, as well as the impact of such demands on the time course of feature space availability as indexed by vERPs. Lastly, the project will test for a potential causal role of vERPs to categorization through the use of advanced neuromodulation techniques.
