A large number of data processing systems---including NoSQL systems, column-stores, MapReduce, data stream managers, complex event processors, in-memory databases, and others---have been developed in recent years. It can be hard for an application developer to pick one system when the needs of her application match the features of multiple of these systems. The choice becomes considerably harder when different components of an application fit the features of different systems. Considerable manual effort goes into creating and tuning such multi-system applications. Current applications are coupled tightly with individual systems, which makes dealing with changes to application execution requirements or workload characteristics nontrivial. Moreover, an application's data and workload properties may change over time, often in unpredictable and bursty ways. Consequently, the system that is best for an application can change over time. Adapting to change can be hard when application development is coupled tightly with any individual system.
This project will design, implement, and evaluate a new breed of Database Management Systems that is termed DBMS+. While conventional Database Management Systems (DBMS) support applications with a single execution engine and a storage engine, a DBMS+ integrates and manages multiple compute and storage systems. An application specifies its execution requirements on aspects like performance, availability, consistency, change, and cost to the DBMS+ declaratively. For all requests (e.g., queries) made by the application, the DBMS+ will select the execution plan that meets the application's requirements best. A novel aspect of the execution plan in a DBMS+ is that the plan includes the selection of one or more specialized systems. The optimizer in the DBMS+ is responsible for determining the most suitable execution plan, which includes choosing the most suitable compute and storage systems for a given application and its requirements. The DBMS+ approach has the potential to address the hurdles that application developers and system administrators face from the vast and growing number of specialized systems.
For further information, see the project web site at: www.cs.duke.edu/starfish/cyclops/
