Visual cognition is orchestrated by the interaction of 'bottom-up' (feed-forward) processes that carry sensory information and 'top-down' (feed-back) processes that modulate the incoming input in the context of goals, tasks, emotions and stored information. At the anatomical level, each area within the cerebral cortex is heavily innervated by both feed-forward signals and feed-back signals. With funding from the National Science Foundation, Gabriel Kreiman, Ph.D. of Children's Hospital Corporation (Boston, Massachusetts) in collaboration with Andreas Schulze-Bonhage, Ph.D., of the Freiburg University Hospital (Freiburg, Germany), is investigating the dynamical integration of bottom-up and top-down neural signals, by combining computational models and machine learning techniques for data analysis with high-resolution neurophysiological recordings from the human temporal lobe. Researchers have long recognized that top-down and bottom-up signals play a key role in visual recognition, however, the relative contribution and interactions between these signals remain unclear. The research project is focused on a particular aspect of cognition, namely our ability to visually recognize patterns, which is central to most everyday tasks. Even the best machine computational models available today only provide a coarse approximation to the complex neurophysiological responses found in higher visual cortex. Not surprisingly, a three-year-old can outperform sophisticated computational algorithms in recognition tasks, such as navigation in complex environments or recognizing objects in cluttered scenes. The research project focuses on three progressively more complex tasks that rely increasingly on top-down influences. The first research aim involves top-down influences during recognition of objects in a cluttered visual stimulus. The second aim examines whether neurophysiological responses in the human temporal lobe can support recognition from partial object information. This question is being approached through studying the phenomenon of object completion. The third aim combines visual stimulus clutter and occlusion in a complex realistic recognition scenario. For this aim, the researchers are examining the influences of attention and task-related goals on neurophysiological activity while epilepsy patients play a custom-designed video game. These neurophysiological data take advantage of the rare opportunity to combine high-resolution neurophysiology, computational models, and behaviorally complex tasks to carry out research that would be difficult with non-human animals.
By furthering the understanding of the transformation of perceptual information into cognition, the researchers are contributing to two broader goals: The goal to help alleviate the challenging conditions involved in cognitive disorders through the development of interfaces between brains and machines, and the goal to apply knowledge about neuronal circuits to develop computational algorithms that automatically extract cognitive information from sensory data. Building a fast, robust, and reliable artificial vision system would have profound repercussions in many areas of science and engineering, including pattern recognition, surveillance and security, automatic navigation and clinical image analysis. These scientific and engineering advances could in turn translate into important real-world applications of interest for industrial partnerships. Understanding the visual system relies on many skills ranging from computer science to physics, neuroscience, and psychology. The research efforts are complemented by educational and outreach initiatives aimed at training interdisciplinary scientists. The training is producing multidisciplinary students who can build on their fundamental scientific skills and apply this knowledge to challenging clinical and engineering problems. This project is jointly funded by the Cognitive Neuroscience Program, the Social Behavioral and Economics Division, Collaborative Research in Computational Neuroscience, and the Office of International Science and Engineering. A companion project is being funded by the German Ministry of Education and Research (BMBF).
Our remarkable ability to rapidly recognize faces, objects and complex shapes is critical to almost every visual task including reading, navigation and socialization. In this project, we have combined computational modeling and invasive neurophysiological recordings from the human ventral visual stream to investigate the neural circuits and algorithms orchestrating visual recognition. One of the remarkable aspects of vision that we have investigated involves object completion from partial information while being occluded images. We have provided behavioral, physiological and computational evidence that suggest that object completion relies on feedback signals that contain prior knowledge and are instantiated by top-down connections. An improved understanding of the neurophysiological mechanisms involved in human visual perception is not only an important step in basic science and engineering but it can also contribute to technical and medical progress. The biologically inspired computational algorithms developed as part of these efforts will help improve the design strategies of artificial intelligence systems involved in pattern recognition, improve technical solutions for machine-based visually guided exploration in terms of sensitivity, specificity and robustness of object recognition, and also help in strategies to design rehabilitative programs to improve human visual perception in patients with impairments in higher order visual discrimination. Another aspect of this project has involved collecting large amounts of high-resolution neural data from diverse brain areas and developing the computational algorithms to infer putative connections and interactions among brain areas. Ultimately, computations in the brain rely and are instantiated by the complex web of interactions among specialized sub-circuits. Previous efforts have used non-invasive techniques to try to predict connectivity patterns. Our efforts take advantage of a rare opportunity to investigate brain function in an invasive way and therefore provide a stronger link to functional networks. This human brain connectome map holds the potential to transform both how we model and simulate brain computations and also how we understand the myriad of neurological conditions that lead to aberrant connectivity patterns. Upon publication, the data and algorithms will be shared with the research effort via a publicly accessible database. The research endeavors are complemented with efforts aimed at disseminating the results and training the next generation of researchers through a variety of mechanisms. As part of this project, the PI has taught several classes at Harvard that are directly related to the investigations, organized conferences and workshops and published multiple journal articles and two books. The laboratory has hosted and trained multiple students (undergraduate students, graduate students) and postdoctoral researchers, including women and minorities, from several institutions both in the Boston area as well as other states and other countries. These research experiences have constituted the initial steps into science for many of those researchers, who have now gone on to continue their education, work in industry or establish their own independent research laboratories.
