Vision is one of the most crucial senses for humans when evaluating a scene. Natural imaging systems such as the eye provide an aberration-free image with a wide field of view. Artificial imaging systems can be made lighter, smaller, and with minimal optical aberrations by curving the focal plane array to match the curvature of the lens. Such a conformal architecture is analogous to the shape of the human eye. Furthermore, integration of an artificial synapse in image sensors will enable to processing of the high volumes of data acquired from a scene at low power. This approach is inspired by the interaction between the eye and visual cortex. In this project, the PI will design integrated hemispherical image sensors with an embedded neuromorphic chip that allows edge computing at hardware level with low-power operation. The system will provide wide-angle, aberration free images for various applications such as vehicle navigation, threat detection, and object identification. This will greatly facilitate further development of broad smart sensor applications combined with simplified AI technology. The PI will create introductory lab modules that connect basic material science and device physics to system level integration, allowing students to connect basic science with real-life applications.
In this program, the PI will demonstrate an artificial retina integrated with neuromorphic circuits that consumes extremely low power. The PI will focus on designing and fabricating a thin-film InGaAs based hemispherical focal plane array and memrisror based artificial synapses that provides a wide field of view, simultaneous localization and mapping (SLAM), data reduction, ranging capability adaptable to variable and near infrared light levels while providing object recognition capabilities. This will be enabled by a combination of various unique technologies; from materials growth and lift-off, flexible device fabrication to heterogeneous and system level integration. The fabricated thin-film compound semiconductor based hemispherical image sensor array using remote epitaxy technique will be integrated with novel computing architecture based on emerging non-volatile resistive switching devices, providing an ideal implementation of a synaptic weight in artificial neural networks. This integrated image sensor will enable edge extraction by applying hexagonal kernel to the honeycomb image sensor array, and object recognition via a single layer of fully connected convolution neural network (CNN) similar to the human retina and visual cortex. Eventually, this proposed project will provide imagers that would enhance situational awareness and recognition with minimal power consumption in a power-constrained environment that is of central importance to robotics, autonomous driving, and military applications etc.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
