This proposal seeks to develop technology for the high performance streaming, merging, filtering and mining of high volume data streams from multiple sources. The application space is targeted at the Sloan Digital Sky Survey (SDSS) and National Virtual Observatory data sets. The work involves developing, extending existing technologies, and applying protocols and services to scale to existing high speed networking infrastructure to support the processing of these high volume data streams. The project will integrate capabilities into the SDSS education project, Skyserver. Impact and intellectual merit lie in the proposed work's direct enabling of SDSS and NVO data sets, and will apply just as well to a set of homeland defense applications and other science domain data. The work will also explore the use of these "teraflow" techniques in high volume transactional backup scenarios across high bandwidth-delay paths.
