The goal of this research is therefore to develop modular middleware and programming technology to address these challenges of grid computing: decentralized coordination, resource optimization, and programmability.
The project will pursue research in the following: (1) it will provide efficient highly-customizable (easily programmable) decentralized coordination of pervasive information and computational services and agents, (2) it will investigate the use of physical and computational agents that are heterogeneous, both in terms of computation and communication capabilities, (3) it will fully explore resource optimization techniques considering the dynamic nature of grid environments, in which nodes can arbitrarily join and leave, and (4) it will apply developed techniques and software to scientific problems in large-scale and open grid computing environments. The proposed middleware will achieve decentralized coordination through efficient peer-to-peer communication protocols, global resource optimization through work-stealing and static and dynamic program analyses, and programmability through highly expressive programming and coordination abstractions and enabling easier and more effective applicability of grid computing technology to solve complex scientific problems executing on scalable high-performance computing or grid computing environments.
The project also includes a comprehensive integrated education and outreach plan, such as the development of a special exhibit at the Troy Junior Museum to introduce high-performance computing to pre-college students from a largely low-income minority community, and the creation of a hands-on laboratory for grid computing, open source software development efforts, and curricular development of undergraduate and graduate courses on distributed computing over the Internet.
This award was used to investigate: (1) programming models, languages, and tools to facilitate development of grid and cloud computing applications, (2) decentralized middleware algorithms and protocols to adapt distributed systems to dynamic run-time environments, and (3) application of programming technology and middleware to scientific disciplines, in particular, astronomy. In addition, the following educational activities were carried out: (1) development of a graduate course on "distributed computing over the Internet", (2) writing of a graduate textbook called "Programming Distributed Computing Systems: A Foundational Approach" to be published by MIT Press, and (3) graduation of three Ph.D. students and five M.S. students as a result of research performed in relationship to this project. In collaboration with IBM Research and Google, we are developing a fault-tolerant distributed programming model, called transactors. Transactors can help future programmers develop robust correct distributed applications executing over unreliable infrastructure at a higher level of abstraction. Our research also resulted in the notion of malleability---the ability to dynamically split or merge computational entities (e.g., actors or processes)---which was proven to be a significant reconfiguration mechanism that helps scale up distributed applications. We also investigated partial redundancy techniques applicable to evolutionary algorithms executing over large-scale heterogeneous volunteer computing environments. These techniques avoid expensive re-computations in the face of potentially malicious users, significantly improving the efficiency of volunteer computing environments, while preserving global computation correctness. One of the key projects that resulted from this grant is the MilkyWay@home volunteer computing project hosted at Rensselaer Polytechnic Institute. As of January 2011, our MilkyWay@home project had 87,742 volunteers from 192 countries donating us 169,274 (typically multi-processor and GPU) hosts or the equivalent of 1.7 petaflops for these analyses. We are using the large computing capacity of MilkyWay@home to evolve different galaxy models (using N-body simulations) which is computationally very expensive and would be impossible without this volunteer computing infrastructure. The ultimate scientific goal is to understand current structure of the Milky Way, to discover potential dark matter distribution and to understand evolutionary pathways of our galaxy. Additional research was performed in mobile distributed actor garbage collection, distributed systems visualization, and offloading computation from mobile phones. In the final period of this grant, we have been investigating the applicability of our programming technology and middleware techniques to the emerging cloud computing paradigm. Please visit the Worldwide Computing Laboratory's web page at http://wcl.cs.rpi.edu to download free open-source code and documentation describing all our software projects.
