This award, to deploy a unique, large, shared memory machine to the science and engineering community, will dramatically increase the size of a heavily oversubscribed TeraGrid resource. The new resource, an SGI UV system with two direcly addressible 16 TB memory banks will be used by a significant group of researchers whose codes cannot efficiently run on the large distributed-memory machines. The SGI UV system will increase the available cycles by more than a factor of 7 compared to current resources.
The Pittsburgh Supercomputing Center (PSC) acquired, installed, developed and put into operation, for the benefit of the national research community, a large shared-memory computing environment comprising an SGI UV-1000 computer, ancillary data storage system and network connections. Named Blacklight, the computer comprises two very large memory spaces which retain the ability to run single applications across the full system. Blacklight consists of two linked SGI UV-1000s, each with 16TB (terabytes) of coherent shared memory. When installed, it involved several new technologies, including the Nehalem-EX processor from Intel and the SGI Hub chip and router. Blacklight differs substantially, both architecturally and in the types of applications that it will support, from the large distributed memory systems in the NSF cyberinfrastructure. Users took advantage of Blacklight’s large shared memory. E.g., jobs that used more than 2TB, which couldn’t have been done elsewhere in the NSF cyberinfrastructure, were common (average of 5/week). In July 2014 there was a job that used 15TB and ran for 33 hours. In total, there were 1,235 projects on Blacklight that used 88 million hours of computing time. Genomics researchers have found Blacklight to be very useful. Rebecca Duncan ,University of Miami, said, "My research in functional and comparative genomics involves a lot of [memory intensive] computation … Although my university has a high performance computer, it did not meet the memory requirements needed for some steps of my assembly pipeline. Blacklight enabled me to overcome the difficulties …" Matthew MacManes and colleagues at the University of California found in a genomics study of mice that lifestyle has impact on bacteria within the female reproductive tract; promiscuous deer mice have 2x bacterial diversity and better diversifying selection for immunity to disease than monogamous California mice. In chemistry, Brian Space and colleagues at the University of South Florida and King Abdullah University of Science and Technology made a promising breakthrough in technology for capturing the greenhouse gas CO2 before it enters the atmosphere. In finance, Mao Ye and colleagues at the University of Illinois found results that led to substantial changes in reporting regulations in odd lot and high frequency trading in the NASDAQ and New York stock exchanges. In addition, researchers achieved significant results in many other fields including machine learning applied to kidney exchange for kidney matching. Graphics for these example scientific achievements accompany this report. More articles on scientific results obtained by users of PSC’s supercomputers are given at www.psc.edu/science/. PSC is active in education, outreach and training. In 2013-2014 alone, there were 2,566 participants in 39 education, outreach and training events that covered about 375 hours of instruction/lab time. PSC has several science education programs, most aimed at K-12 and undergraduate teachers and some aimed at students. Better Educators of Science for Tomorrow prepares teachers to introduce basic ideas in bioinformatics and to raise awareness of emerging biomedical careers. It provides a complete high school level bioinformatics curriculum. Computational Modules in Science Teaching brings innovative science tutorials into secondary school and college classrooms. The Computation and Science for Teachers Professional Development Program is an integrated set of modules to train teachers on how to incorporate computational reasoning and tools such as modeling and simulation into their middle and high school math and science curricula. PSC staff members also created many new technologies. Just three are discussed here. PSC people have obtained a patent for ZEST, a highly scalable parallel file system designed for maximum efficiency with write-intensive application workloads such as checkpointing. SLASH2 is a distributed filesystem that incorporates existing storage resources into a common filesystem domain. It provides system-managed storage tasks for users who work in widely distributed environments. The SLASH2 metadata controller performs inline replication management, maintains data checksums and coordinates third party, parallel, data transfer between constituent data systems. The SLASH2 I/O service is a portable, user-space process that does not interfere with the underlying storage system's administrative model. Data Supercell is a high-performance (high bandwidth, low latency), high-reliability, high-capacity, low-power storage system for large data storage requirements including but not limited to near-line storage, data warehousing, large-scale data analytics, and archiving. Using SLASH2, the Data Supercell couples appropriate hardware and software technologies in a unique way to deliver valuable functionality for meeting large data handling requirements. Patents are pending for SLASH2 and data Supercell.
