This MRI award will serve to purchase a 256 Processor/1536 core SGI Altix UV1000 with 9.2 TB of RAM and 14.4 TB of raw scratch disk space. The instrument will provide scientists at Princeton University, the Institute for Advanced Studies, and partner institutions with the computational resources needed to model multi-scale phenomena in the sciences and engineering. The flexibility of the Altix architecture, which supports both shared and distributed memory applications, along with an outstanding bus architecture to support the addition of extra processing units such as GPGPUs is an ideal platform for developing algorithms for multi-scale problems. The setting of the instrument in the University?s High Performance Computing Research Center will facilitate a cross-disciplinary approach combining expertise in applied mathematics, computer science and domain-specific disciplines enabling innovative approaches for memory intensive applications. The new instrument will play an essential role in educating a new generation of scientists and training students across many disciplines in the use of advanced modeling tools on modern computer platforms, contributing to new graduate student certificate programs offered by PACM, the Program in Applied and Computational Mathematics, and PICSciE, the Princeton Institute for Computational Science and Engineering. Finally, the instrument will provide a necessary link between local and national facilities, preparing the Princeton scientific community to the emerging multicore and massively parallel architectures of the future.
The instrument will enhance international scientific cooperation by contributing to projects like the Munich-Princeton collaboration in cosmological computational science, and will contribute to science education of the general public through collaboration with the American Museum of Natural History in New York City, with planned new visualizations for use in the Cosmos series and in conjunction with the Museum?s ongoing public education work on earthquakes and geologic movement. Women?s participation in computational projects enabled by the instrument will set examples to encourage greater access of women to science. Finally, access provided to partners at California State University-Northridge will contribute to training minority scientists and engineers.
This award led to the acquisition of a state-of-the-art shared memory computer, an SGI UV1000 system with 1536 cores, named Hecate. This system added significant new capabilities to the Terascale Infrastructure for Groundbreaking Research in Science and Engineering (TIGRESS), a suite of hardware platforms accessible to the entire University community. The TIGRESS facilities are operated at the High Performance Computing Research Center in the new Forrestal Campus of Princeton University. Hecate has become accessible to Princeton University and its institutional partners, the Institute for Advanced Study (IAS), the Princeton Plasma Physics Laboratory (PPPL), the American Museum of Natural History (AMNH), and the University of Munich, in the first quarter of 2012 and has been used extensively since then: 103 registered researchers from 20 faculty groups have used the system in 2013. Hecate has been used for training and research purposes. On the training side the platform served to educate graduate students and postdoctoral researchers in the use of computational algorithms on modern multi-core computer architectures. This activity was corroborated by well-attended interdisciplinary courses offered by the Program in Applied and Computational Mathematics (PACM) and the Princeton Institute for Computational Science and Engineering (PICSciE) in the context of a new graduate program certificate. On the research side the platform has been used to develop new algorithms, to run medium- to large-scale numerical simulations, to foster the expertise needed to run very large-scale simulations at national supercomputing facilities, and to analyze and visualize the vast amounts of data produced in the simulations. Hecate has made possible research advances in several disciplines, ranging from computational chemistry, to fluid dynamics, computational seismology and computational cosmology. Some highlights are illustrated in the attached images. (1) In a joint study, the experimental group of Ulrike Diebold at Vienna University and the theoretical group of Annabella Selloni at Princeton University have shown that a form of titanium dioxide called anatase is an effective oxidation catalyst due to a sequence of atomic displacements triggered by oxygen adsorption. These processes bring a subsurface lattice vacancy to the surface as shown in the frames (A to F) in the attached figure, in which O atoms near the defect (purple, aqua, and blue) shift positions. Those steps result in O2 incorporation into the crystal surface. The atomic motions revealed by ab-initio molecular dynamics simulations were confirmed by beautiful scanning tunneling microscopy (STM) experiments. (2) Ab-initio molecular dynamics simulations by Roberto Car, Annabella Selloni and postdoctoral researcher Li-Min Liu (now at the Bejing Computational Science Research Center) of the Princeton Chemistry Department revealed the atomic mechanisms by which functionalized graphene sheets (FGS) dispersed in liquid nitromethane greatly enhance the burning rate of this monopropellant fluid. This process is illustrated schematically in the attached figure, which depicts proton transfer events between a nitromethane molecule or a nitromethane fragment produced in the combustion process and FGS. Hydrogen, oxygen, carbon, and nitrogen atoms are white, red, cyan, and blue, respectively. These simulations helped to understand experimental observations by Ilhan Aksay and Daniel Dabbs of the Princeton Chemical Engineering Department, and by Richard Yetter of the Mechanical and Nuclear Engineering Department of the Pennsylvania State University. This study has technological implications for the development of new fuels for propulsion systems. (3) Ostriker and PhD student Ena Choi have been studying how massive black holes, which are known to exist at the centers of galaxies, will modify the evolution of these systems. They find that when gas flows reach the centers, the black holes will accrete some of this gas, and the accreted gas emits radiation that can drive enormous winds out of the centers of these galaxies, sometimes stripping them bare and often driving holes in the surrounding intergalactic medium. The effects are especially dramatic during galaxy mergers and the attached picture taken, from a high resolution, hydrodynamic computer simulation of a merger or two Milky Way type galaxies, shows the hot gas streaming out of the galaxies just after they have passed through one another and the black holes erupted violently. Research advances such as those described above and others required new algorithms and/or improvement of existing algorithms to harness the power of modern computational platforms like Hecate. These algorithmic developments have being made available to the international science and engineering community through open software platforms and through participation of Princeton scientists in workshops and tutorials on the international scene. Visualizations made possible by the instrument, such as those modeling galactic evolution, have been used to educate the general public in the Cosmos series of the AMNH. Access to partners at California State University-Northridge and to undergraduate students visiting Princeton University in summer in the context of the REU (Research Experiences for Undergraduates) program of the NSF has further contributed to the broader impact of this instrument acquisition.
