Logo des Repositoriums
 
Zeitschriftenartikel

Preserving Recomputability of Results from Big Data Transformation Workflows

Vorschaubild nicht verfügbar

Volltext URI

Dokumententyp

Text/Journal Article

Zusatzinformation

Datum

2017

Zeitschriftentitel

ISSN der Zeitschrift

Bandtitel

Verlag

Springer

Zusammenfassung

The ability to recompute results from raw data at any time is important for data-driven companies to ensure data stability and to selectively incorporate new data into an already delivered data product. However, data transformation processes are heterogeneous and it is possible that manual work of domain experts is part of the process to create a deliverable data product. Domain experts and their work are expensive and time consuming, a recomputation process needs the ability of automatically adding former human interactions. It becomes even more challenging when external systems are used or data changes over time. In this paper, we propose a system architecture which ensures recomputability of results from big data transformation workflows on internal and external systems by using distributed key-value data stores. Furthermore, the system architecture will contain the possibility of incorporating human interactions of former data transformation processes. We will describe how our approach significantly relieves external systems and at the same time increases the performance of the big data transformation workflows.

Beschreibung

Kricke, Matthias; Grimmer, Martin; Schmeißer, Michael (2017): Preserving Recomputability of Results from Big Data Transformation Workflows. Datenbank-Spektrum: Vol. 17, No. 3. Springer. PISSN: 1610-1995. pp. 245-253

Zitierform

DOI

Tags