We propose a """"""""data commons"""""""" to provide data storage for seven core facilities that produce or analyze large and complex datasets for biomedical researchers at Duke University and their collaborators worldwide. With data storage requirements already counting in hundreds of terabytes the core facilities continue to adopt new technologies that will further increase the flows of data by orders of magnitude, a looming challenge for data retention and analysis. Our proposal seeks to turn this excess of data from a challenge to strength for life sciences researchers. Equipment purchased with the NIH SIG grant will use a mixture of disc arrays (~450 usable terabytes) and tape library with initial uncompressed data capacity of 1 petabyte to create a scalable data storage resource with features that protect and preserve data and manage data efficiently. Disc arrays will be configured to serve up data to existing computational servers for analysis either by researchers or by staff in Duke's """"""""Omics Analysis Core Facility,"""""""" which was created in 2012 to provide support to researchers unfamiliar with using genomic data sets. The disc arrays and tape library will function in a coordinated fashion in a Quantum """"""""Stornext"""""""" file system, and data management policies and system usage data collection will allow for fine-grained optimization of system performance and economy. The Quantum systems have been widely deployed in data-intensive industry and research, including in the genome sciences. From the outset, the core facilities will use the equipment to enhance their integration. Data provenance and management features of the Duke's Express Data Repository - used by the proteomics and microarray core facilities since 2006 - will be attached to the proposed storage equipment so that services can be expanded to include sequence data, RNAi screening data, microscopy images, and results from the analysis core. Thus, the proposed data commons will function in a manner allowing for efficient and, in many cases, automatic data hand-offs within a protected storage framework. This integration removes logistical impediments for data integration, reduces chances for accidental (or malicious) data corruption, and enhances the scope of automation for large-scale research projects. The storage will be connected via the cores to data-producing equipment and to computational servers run by the 'omics analysis core and to storage on a high-performance analysis cluster. All IT assets, including the proposed equipment, are administered by professional IT staff with particular strength in infrastructure required for large-scale research in the genome sciences. While increase in efficiency of data management will be an immediate benefit, the more important goal of the project is to enable researchers to speed their use of integrated and complex data to explore the complexity of human health and disease.
Showing the most recent 10 out of 154 publications
