Logo des Repositoriums

Autonomous Data Ingestion Tuning in Data Warehouse Accelerators


Volltext URI


Text/Conference Paper





ISSN der Zeitschrift



Gesellschaft für Informatik, Bonn


The IBM DB2 Analytics Accelerator (IDAA) is a state-of-the art hybrid database system that seamlessly extends the strong transactional capabilities of DB2 for z/OS with very fast processing of OLAP and analytical SQL workload in Netezza. IDAA copies the data from DB2 for z/OS into its Netezza backend, and customers can tailor data maintenance according to their needs. This copy process, the data load, can be done on a whole table or just a physical table partition. IDAA also o ers an incremental update feature, which employs replication technologies for low-latency data synchronization. The accelerator targets big relational databases with several TBs of data. Therefore, the data load is performance-critical, not only for the data transfer itself, but the system has to be able to scale up to a large number of tables, i. e., tens of thousands to be loaded at the same time, as well. The administrative overhead for such a number of tables has to be minimized. In this paper, we present our work on a prototype, which is geared towards e ciently loading data for many tables, where each table may store only a comparably small amount of data. A new load scheduler has been introduced for handling all concurrent load requests for disjoint sets of tables. That is not only required for a multi-tenant setup, but also a significant improvement for attaching an accelerator to a single DB2 for z/OS system. In this paper, we present architecture and implementation aspects of the new and improved load mechanism and results of some initial performance evaluations.


Stolze, Knut; Beier, Felix; Müller, Jens (2017): Autonomous Data Ingestion Tuning in Data Warehouse Accelerators. Datenbanksysteme für Business, Technologie und Web (BTW 2017). Gesellschaft für Informatik, Bonn. PISSN: 1617-5468. ISBN: 978-3-88579-659-6. pp. 531-532. Industrial Program - Data Lake. Stuttgart. 6.-10. März 2017