This Sensor proposal focuses on the development of sensory information processing front-ends for a minimally-attended visual surveillance network. We intend to use mixed-signal hardware computation methods in a system-on-a-chip architecture. Image processing algorithms are implemented at the focal-plane, using compact circuits that operate at microwatt power levels to minimize battery weight and form factor. Furthermore, the sensory front-end must demonstrate operational autonomy (i.e. in decision making) and robustness (i.e. to changes in environmental conditions). Intellectual Merit: The proposed system will replace traditional computer vision systems (cameras, digitizers and processors) with a computational sensor. Using wide-dynamic range imaging, with local gain control, circumvents the limitations of standard cameras. Spatiotemporal feature extraction is used to highlight moving targets. The shape formed by the features is identified using kernel learning in silicon; alarms are generated based on the similarity of the targets to preprogrammed shapes. Without these types of smart, ultra-low power, compact imaging and computational microsystems, practical sensor networks for visual surveillance may not be realizable. Broader Impact: The sensors are primarily intended for surveillance of large, remotely located areas, where limited manpower is available, e.g. border patrolling. Their low power and small size obviates a variety of mobile applications. The multi-disciplinary nature of this work will result in the development a pipeline of students, educators and researchers with the broad skills required to succeed in modern high technology industry and academics. Their education will be rounded with exposure to issues in homeland security, privacy rights and international law.
