Overview: Deep neural networks (DNN) have become the de-facto standard tool to carry out complex learning tasks. DNNs belong to the second generation of artificial neural networks (ANNs), which rely on neurons that implement memory-less non-linear transformations of the synaptic inputs. Motivated by the biological analogy with the behavior of neurons in the brain, the third generation of neural networks, also referred to as Spiking Neural Networks (SNNs), was introduced in the nineties. In SNNs, synaptic input and neuronal output signals are spike trains. This proposal argues that the time for the use of SNNs as machine learning tools has come, and sets forth a systematic approach for the design and implementation of SNNs as learning and inference machines.
Intellectual merit: SNNs have a number of unique advantages as compared to ANNs: (i) They are event-based systems with natural sparsity properties, which have the potential to make deep learning machines feasible for energy-limited devices; (ii) They are uniquely capable to natively process data that comes in the form of timeencoded processes, for example, from bio-inspired sensors. The main goal of this project is the establishment of a theoretical framework to enable the design of flexible spike-domain learning algorithms that are tailored to the solution of supervised and unsupervised cognitive tasks, as well as their co-optimization on nanoscale hardware architectures. To this end, this project puts forth a principled probabilistic framework based on the graphical formalism of Directed Information Graphs.
Broader impact: The outcome of this research is expected to have a profound impact on the increasing number of practical applications that are based on the processing of time-encoded signals, including biological sensors and next-generation communication systems, and/or that require the adoption of computing solutions with a significantly smaller power budget as compared to conventional DNNs. The research methodology is based on a multi-disciplinary approach that integrates machine learning, information theory, probabilistic graphical models, neuromorphic computing and device/system architecture at the nanoscale. The educational plan at the home institution targets both undergraduate and graduate students via hands-on learning and experimentation activities.
