In this proposal the authors introduce a new concept for an optical device, the optical Rectifying Linear Unit or ReLU, and motivate it as the key enabling technology required for constructing an optical deep learning system. The ReLU neuron was introduced to alleviate the vanishing gradient problem in deep multi-layer neural networks, and it has allowed the training of state-of-the-art deep neural networks.
The authors present an efficient optical implementation of the ReLU in which a bidirectional transmissive optical switch that is controlled by an interferometric differential detector part of the forward propagating field implements both the rectifying linear forward response but also the derivative needed for gating the backwards propagating error needed for deep learning. The optical ReLU is the key component necessary to realize the vision of a physically implemented trainable optical machine learning technology.
Such an optical deep learning system has the potential to scale up in performance to a level far in excess even the most optimistic projections for the further development of massively parallel super computers and will use much less energy by harnessing the efficient analog computational capabilities of coherent photons. The research team will design, fabricate, test, and demonstrate large arrays of these new optical ReLU devices using liquid-crystal-on-Silicon smart-pixel technology. A proof-of-concept laboratory demonstration of a self-aligning deep learning optical system will then be developed.
This project will train a graduate student in the fields of machine learning and deep neural networks as well as CMOS device design and fabrication, liquid crystal chemistry and physics, coherent and nonlinear optics, lasers, and computer controlled experimental technology. Such a cross-disciplinary background will produce a nimble research leader capable of advancing the frontiers of both optical and machine learning science and technology. During this program the PI will continue to develop a Massively Open Online Course (MOOC) in the areas of Fourier optics and holography that will help to train a new generation of optical scientists.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
