Datacenters and clouds have become the main compute platform for many large scale corporations. Petabyte-scale datasets are stored throughout the datacenter and hundreds to thousands of different users submit jobs to collect business intelligence, gather statistics, or to compute essential data, such as a large scale index or a list of top users or their message posts. However, a significant challenge to these centers arises from user manipulation by some or all of the many tenants (both intentional and unintentional) of the underlying resource allocation mechanisms through misreporting of true job characteristics.
This project is designing and testing new algorithms combining ideas from computer science and economics which will make datacenters and clouds more robust to user manipulations. It is initially focusing on two important abstractions. The first allocates the total amount of resources and the second allocates individual machines (or virtual machines). The former is the standard abstraction for datacenters and the latter for many clouds, such as Facebook's internal cluster and Amazon's public EC2. For the former, where there are existing non-manipulable algorithms, the PIs are developing extensions to support new requirements, such as jobs with constraints. For the latter, the PIs are inventing new non-manipulable algorithms based on their preliminary studies. In addition, this project is applying recent results from algorithmic mechanism design and game theory to develop general procedures for converting existing manipulable protocols into non-manipulable ones.
This project will lead to more robust mechanisms for datacenters and clouds, reducing costs and energy usage.
