The Dark Energy Survey (DES) is an intermediate term survey, in the years 2011-2016, that will address the nature of dark energy using four independent and complementary techniques: (1) a galaxy cluster survey in collaboration with NSF?s South Pole Telescope Sunyaev-Zel'dovich Effect mapping experiment, (2) a cosmic shear measurement, (3) a galaxy angular clustering measurement within redshift shells to z=1.35, and (4) distance measurements to approximately 2000 type 1a supernovae (SNe Ia). The DES will produce about 170Tb of raw data, processed into science-ready images and catalogs and co-added into deeper, higher quality images and catalogs. In total, the DES dataset will approach 2Pb. This data volume and the survey duration require a new type of data management system with a high degree of automation and robustness, including integrated quality assurance testing. The present collaborative project will build the DES Data Management (DESDM) system. DESDM has been designed and tested, using both simulated data and real data from the Mosaic2 camera. The project team has created working prototypes of many elements of DESDM and will now transform those prototypes into the production version of the DESDM system.
DES is a complex international, multi-agency project, with far-reaching foreign and domestic impacts. It is organized into DESDM and two other themes. These are the Dark Energy Camera (DECam), to be installed at the 4-meter Blanco telescope of the Cerro Tololo Interamerican Observatory (CTIO), and the facilities improvement and integration project at the CTIO (CTIOFP). The CTIOFP is supported by CTIO, a component of NSF's National Optical Astronomy Observatory, and DECam is supported by the Department of Energy (DoE) and by international and institutional partners. DESDM is supported by NSF and also by the project partners.
The DESDM project is working collaboratively with other large scale imaging projects to develop common solutions, and will be making all produced software available to the community. Students will play important role, developing expertise in aspects of scientific data management, high performance computing, and the development of science analysis tools. The DESDM system will archive the DES data, enabling public access and science analyses (after a one year proprietary period) through a provided scientific analysis interface that will use high performance computing resources at the NSF's National Center for Supercomputing Applications (NCSA), operated by the University of Illinois. A separate part of the project not covered by this award is a DECam Reduction Portal, supporting non-DES users of the DECam to reduce their own data.
The main purpose of this NSF Grant is to construct a system to process the data for the Dark Energy Survey, which will be acquired by the DECam instrument on the Bianco Telescope at the Cierra Tololo Observatory (CTIO) in Chile. CTIO is operated by the National Optical Astronomy Observatory. The Dark Energy Survey expects data taking to begin in the fall of 2012. We have constructed a processing system capable of transforming unprocessed data from DECam and processing it into extensive catalogs of data suitable for a Cosmology Survey. Important elements of numerical computation include custom de-trending codes, which remove instrumental signature from the images, and calibration codes, which allow us to very precisely locate the images onthe sky, and know the amount of light emitted by each object. Working closely with the members of the astronomical community, we have integrated and adapted community software for high level cataloging, characterization of detection as stars or galaxies, and developed techniques to add images of the same part of the sky to increase the depth of the survey without distorting detailed features of the images needed for weak lensing and other similar studies. We have also integrated codes for two specific science working groups: a Difference Imaging pipeline for the detection of supernovae, and weak lensing. We have processed simulated data DECam data and actual data from an older camera on the Bianco telescope and feel that the science codes are ready for first light from the DECAM instrument. We expect the total amount of data to be on the order of a few petabytes, with about 100 Terabytes of that being catalog data. The catalog data is the kind of data primarily used by cosmologists studying dark energy. Delivering catalog-based data to scientists is key to enabling the science. Accordingly, we have built our system around a commercial Oracle database, which will scale to serve the catalog queries of the DES scientists, even early on in the survey, when query patterns will be fluid. Our work products have been tendered to a follow-on operations project, which is preparing them for production operations, and he changes needed when actual DECam data is available.
