This project develops efficient machine vision algorithms inspired by the architecture and energetic efficiency of the primate visual system for motion processing. Navigating through a rich cluttered natural environment, while both the observer and the objects in the scene are moving, is a difficult problem in machine vision, particularly for real-time processing under power constraints. However, humans and other animals perform these tasks with ease. The nervous system is under tight metabolic constraints and this leads to incredibly efficient representations of important environmental features, such as the observer's heading, the depth of objects, and the motion of objects. In addition, these efficient machine vision algorithms can be applied to robotics, the IoT, and edge processing. The algorithms can be applied to a wide range of applications, including augmented reality, assistive robotics, autonomous vehicles, and the Internet of Things (IoT) Thus, they could have a transformative economic and societal impact by creating applications that can operate autonomously over long periods in remote locations.
Inspired by ability of the nervous system to efficiently encode and appropriately respond to the visual features that make up a dynamic scene, the algorithm uses sparse predictive coding techniques to process data streams from cameras. Because the algorithms can be realized in spiking neural networks, where the artificial neurons only send signals when an event occurs, they can run efficiently on low powered neuromorphic systems; computers that support such representations. By employing an architecture inspired by the brain, where op-down signals from the frontal cortex and parietal cortex predict where objects will be in the future, the system will have better object tracking and overcome difficulties when objects become hidden from view. These representations are sparse and reduced, leading to energy efficient processing, less computation, and thus low power consumption. In summary, the machine vision algorithms: (1) increase our understanding of how the brain encodes behaviorally relevant signals in the world, (2) lead to computationally efficient handling of large data streams, and (3) realize power efficient processing for a wide range of embedded applications.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
