This project studies the effects of incorporating, into deep neural networks for visual processing, several heretofore unincorporated features of biological visual cortical circuits. Deep neural networks are artificial circuits loosely inspired by the brain's cerebral cortex. Their abilities to solve complex problems, such as recognizing objects in visual scenes, have revolutionized artificial intelligence and machine learning in recent years. The hierarchy of layers in a deep network trained for visual object recognition also provides the best existing models of the hierarchy of areas in the visual cortex implicated in object recognition (the "ventral stream"). This project seeks to understand whether and how incorporating additional features of brain circuits may (1) improve machine learning performance, particularly on tasks that are more challenging than those typically studied; and (2) yield improved models of visual cortex. Improving the performance of deep networks would yield great benefits across wide swaths of society and industry that are impacted by advances in artificial intelligence. Improved models of visual cortex will advance understanding of cortical function, which may lead to significant further benefits for understanding normal mental functioning and perception and their potential enhancement, as well as mental illness and perceptual and cognitive deficits.
Deep networks currently achieve their success using almost purely feedforward processing. Yet the visual cortical ventral stream that helped inspire deep networks also uses massive recurrent processing within each area as well as feedback connections from higher areas to lower areas and "bypass" connections from lower areas to areas multiple steps higher in the hierarchy. Deep networks also use "neurons" that can either excite or inhibit different neurons that they project to, whereas biological neurons are exclusively excitatory or inhibitory. This project will incorporate feedback and bypass connections into deep networks, as well as local recurrent processing in networks of separate excitatory and inhibitory neurons. Recent work by the investigators has shown how local recurrent processing explains a number of nonlinear visual cortical operations often summarized as "normalization." Simple forms of normalization currently used in deep networks maintain activities in an appropriate dynamic range, but the biological forms of normalization involve interactions between different stimulus features and locations in determining neural responses, which may have important computational roles e.g. in parsing visual scenes. The performance of deep networks incorporating these features will be assayed on a variety of visual tasks and as models of ventral stream neural data and human psychophysical data, and compared to performance of existing deep net models.
