HyPer Beyond Software: Exploiting Modern Hardware for Main-Memory Database Systems

Abstract

In this paper, we survey the use of advanced hardware features for optimizing main-memory database systems in the context of our HyPer project. We exploit the virtual memory management for snapshotting the transactional data in order to separate OLAP queries from parallel OLTP transactions. The access behavior of database objects from simultaneous OLTP transactions is monitored using the virtual memory management component in order to compact the database into hot and cold partitions. Utilizing many-core NUMA-organized database servers is facilitated by the morsel-driven adaptive parallelization and partitioning that guarantees data locality w.r.t. the processing core. The most recent Hardware Transactional Memory support of, e.g., Intel’s Haswell processor, can be used as the basis for a lock-free concurrency control scheme for OLTP transactions. Finally, we show how heterogeneous processors of “wimpy” devices such as tablets can be utilized for high-performance and energy-efficient query processing.