The backbone of IT infrastructure is evolving towards a service-oriented model, in which distributed resources (software services, virtualized hardware infrastructure, data repositories, sensors, and network overlays) can be composed as a customized IT service on demand. In particular, cloud computing infrastructure services manage a shared ``cloud'' of servers as a unified hosting substrate for diverse scientific applications, using various technologies to provision servers and orchestrate their operation. At the same time, high-speed networks increasingly offer dynamic provisioning services at multiple layers. Network-connected clouds offer a general, flexible, and powerful model to scale up computing power for data-intensive science applications running at multiple cloud sites. The software produced in this project offers interfaces and control policies for application-driven orchestration of federated clouds interconnected by advanced networks.
The project develops software to link cloud computing clusters to other cyberinfrastructure resources through dynamically provisioned networks. A principal focus is to extend popular cloud infrastructure software with hooks to connect provisioned machine instances running in the cloud to external resources through dynamic circuit networks. The project enables cloud applications to dynamically request compute resources at multiple points in the network, together with bandwidth-provisioned network pipes to interconnect them and link them with other services and data repositories. The orchestration framework is based on the Open Resource Control Architecture (ORCA), an extensible platform for dynamic leasing of resources in a shared networked infrastructure. The resource allocation policies are enabled through semantic resource descriptions and extended intelligent SPARQL queries. Driving applications for this project are MotifNetwork, IMG/JGI and Supernova Factory and sensor networks linked to cloud resources (CASA). Development and demonstrations leverage the Breakable Experimental Network (BEN, a multi-layer optical network testbed located in North Carolina), NLR and ESNet.
The goal of this project was to develop software to connect virtual machines running in the cloud to external resources such as sensors via dynamically provisioned networked links. Data from sensors such as radars needs to be transmitted, archived and processed by back-end servers and our project focused on demonstrating the feasibility of using dynamically provisioned cloud and network resources for such applications. We developed a distributed radar application called Nowcasting, and its cloud-based counterpart called CloudCast to demonstrate the work-flows required for high-bandwidth sensing applications running in the cloud. Nowcasting and CloudCast provide highly-personalized short-term weather forecasts based on a user's current location, and unlike traditional web-based cloud application, such application have tight timeliness requirements and also need high-bandwidth needs. Through building application work-flows, our project demonstrated that today's commercial and research clouds are able to successfully support several classes of high-bandwidth sensing applications. In doing so, our project demonstrated the ability to host such applications using dynamic cloud resources in a more cost-effective manner. The broader impacts of this project have been the design of a cloud-based application to provide highly-personalized weather forecasts to end-users, the demonstration of such an application to the community at GENI conferences, and the mentoring of a graduate student and a post-doctoral researcher.
