The development of deep convolutional networks (DCNs) has recently led to great successes in machine vision. Despite these successes, to date, the most impressive results have been obtained for image categorization tasks such as indicating whether an image contains a particular object. However, DCNs ability to solve more complex visual reasoning problems such as understanding the visual relations between objects remains limited. Interestingly, much work in computer vision is currently being devoted to extending DCNs, but these models are still outmatched by the power and versatility of the brain, perhaps in part due to the richer neuronal computations available to cortical circuits. The challenge is to identify which neuronal mechanisms are relevant and to find suitable abstractions to model them. One promising set of candidates is the neural oscillations that are found throughout the brain. This project seeks to identify the key oscillatory components and characterize the neural computations underlying humans ability to solve visual reasoning tasks, and to use similar strategies in modern deep learning architectures.
This project will use existing computational models to develop tasks and stimuli to be used in EEG studies to identify the key oscillatory components underlying human visual reasoning ability. The analysis of these EEG data will be guided by the development of a biophysically-realistic computational neuroscience model. This will inform the development of hypotheses on the circuit mechanisms underlying the oscillatory clusters and relate these mechanisms to neural computations. Finally, the project will develop novel machine learning idealizations of these neural computations, which are trainable with current deep learning methods but still interpretable at the neural circuit level. In particular, the project will further develop initial machine learning formulation of oscillations based on complex-valued neuronal units, thus extending the approach and demonstrating its ability to qualitatively capture key oscillatory processes underlying visual reasoning.
A companion project is being funded by the French National Research Agency (ANR).
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
