We are now in the midst of the big data revolution where enterprise services are increasingly being driven by operational and business models that are powered by data analysis. A key part in making big data successful is ensuring that basic data processing primitives can execute efficiently on large and every increasing volumes of data. However, data processing kernels today largely employ techniques that have been designed about three decades ago, and are now out of touch with modern hardware that has made a fundamental technological shift. First, driven by power consumption characteristics, modern processors now have multiple processing units (called cores) fabricated in a single chip. In contrast, processors just a few years ago were single core. Second, traditionally the storage media for data has been the magnetic hard disk. Now, data has started to move nearly permanently to higher levels of the memory hierarchy, and more specifically to main memory. The goal of this project is to rethink key aspects of data processing techniques for the modern many-core and main memory hardware environment. The research approach is to design, implement and evaluate various methods for data kernels that can be used to store and process data efficiently. In other words, the key focus is on producing data kernels that "run at the speed of modern hardware." Thus, this project aims to have a broad impact on the big data ecosystem by developing faster, cheaper and more energy-efficient data kernels.
