We are entering an Industrial Revolution in the production of information. While in the past data was "handmade" by typing on keyboards, today data are increasingly manufactured by machines: sensors, cameras, software logs, etc. When harnessed in a timely manner, these data can have significant positive impact in many contexts, including early warning and rapid response in natural disasters, air quality monitoring, and improved Internet security. To provide useful information in these contexts, computers in multiple locations must coordinate over networks, because the data are both widely distributed and massive, and cannot be "warehoused" at a single location in a timely manner. Worse, sensor data is typical "noisy" or erroneous in various ways, so statistical methods must be employed to convert the raw "evidence" data into probabilistically reliable information.
In this project we develop new techniques to integrate statistical inference methods from AI with overlay network algorithms developed for peer-to-peer and wireless settings. We design new overlay network algorithms customized for distributed inference. We also develop network-aware inference algorithms that can trade off inference approximation quality for communication efficiency and robustness to network failure. Finally, we explore the use of a high-level declarative language for programming both the networking and inference logic. The high-level language enables us to investigate compilation techniques to co-optimize the inference and overlay network tasks for maximal utility. We prototype and evaluate our ideas via open-source implementations deployed on testbeds like Emulab and Planetlab. Software and research papers are disseminated at http://declarativity.net.
