This project is building cooperating networks of cameras that can be used to reconstruct three-dimensional (3D) features, produce images from novel viewpoints, match trajectories or objects against known patterns, or combine these tasks to provide a powerful, flexible monitoring system.
High data rates and precise calibration requirements present challenges that are not faced by earlier, simpler sensor networks. The bandwidth required to transmit video data from hundreds or thousands of cameras to a central location for processing would be enormous.
Instead, this project is building low-power smart cameras that process video data in real time, extracting features and 3D geometry from the raw images of cooperating cameras. These compressed results, still somewhat bandwidth intensive, are stored in the network until required by users. Content-based routing techniques enable queries against a space-time representation of the data. Query processing occurs in-network, greatly reducing bandwidth requirements.
Camera networks must calibrate precisely, discover and track objects, route view requests to viable cameras, and avoid unnecessary transmissions. Content-routing techniques that will allow cameras to find common features---critical for calibration, search, and tracking. These techniques allow features to be stored and processed near their acquisition point, avoiding wasted communication. Integrated, application-specific compression techniques further reduce overhead.
This project also aims to simplify the engineering effort in building 3D sensornet applications. A space-time ``cube'' abstraction is used to represent the data of the entire sensornet, throughout time. A declarative language is used to specify feature patterns for search and tracking. High-level features can be described as compositions in space-time of simpler features. These descriptions are compiled to use the data-centric protocols to implement data selection, search, or tracking without data centralization.
