Artificial intelligence draws its inspiration from biological intelligence, and both rely on learning to achieve intelligent behavior. Thus, within AI, the field of machine learning plays a central role having indeed achieved impressive successes in dealing with many complex tasks, ranging from computer vision to language understanding, and thereby benefiting billions of humans. Machine learning uses large networks of artificial neurons, which are simplified versions of biological neurons, where learning is implemented by progressively adjusting the weights of the connections between these neurons. The deep learning problem faced by both biological and artificial neural networks is precisely the problem of how deep neurons, located far away from the network inputs or outputs, can adjust their connection weights to ensure that the networks behave intelligently. This fundamental problem and the space of its possible solutions are not well understood. Because deep learning has wide range of applications in technology and science, from computer vision to protein structure prediction, progress in our fundamental understanding of deep learning is likely to have a broad impact across multiple areas. Furthermore, the theory of local learning is inspired by biological considerations and it has the potential for strengthening the bridge between AI/machine learning and neuroscience. The resulting theory, algorithms, data, software, and results will be broadly disseminated through multiple channels and integrated into educational and outreach efforts. The project PI will continue his broad activities bringing research into undergraduate and graduate courses, outreach to local high school students through hosting at a summer program and lectures to high school students.
To try to address the deep learning problem, over half a century ago D. Hebb proposed a vague strategy often summarized by the expression "neurons that fire together, wire together". The essence of this effort is to improve our fundamental understanding of deep learning by bringing clarity to Hebb's proposal and providing a novel rigorous framework for studying learning rules. The framework requires first introducing the notion of local learning: in a physical neural system, learning rules for adjusting connection weights must be local, i.e. functions of only variables that are available locally. Thus one must separate the definition of local variables from the functional form that ties them together into a learning rule. This separation enables the creation of a systematic program for studying local learning rules, by first stratifying learning rules according to their functional complexity, and then studying their behaviors in networks of increasing complexity, from shallow and linear to deep and non-linear. The proposed program of study is likely to lead to the discovery of new learning rules and a better understanding of the capacity and limitations of local learning, ultimately advancing our theoretical and practical understanding of the deep learning problems and its solutions.
