Logo des Repositoriums

Auflistung nach Schlagwort "data integration"

1 - 5 von 5
  • Zeitschriftenartikel
    A distributed data exchange engine for polystores
    (it - Information Technology: Vol. 62, No. 3-4, 2020) Kaitoua, Abdulrahman; Rabl, Tilmann; Markl, Volker
    There is an increasing interest in fusing data from heterogeneous sources. Combining data sources increases the utility of existing datasets, generating new information and creating services of higher quality. A central issue in working with heterogeneous sources is data migration: In order to share and process data in different engines, resource intensive and complex movements and transformations between computing engines, services, and stores are necessary. Muses is a distributed, high-performance data migration engine that is able to interconnect distributed data stores by forwarding, transforming, repartitioning, or broadcasting data among distributed engines’ instances in a resource-, cost-, and performance-adaptive manner. As such, it performs seamless information sharing across all participating resources in a standard, modular manner. We show an overall improvement of 30 % for pipelining jobs across multiple engines, even when we count the overhead of Muses in the execution time. This performance gain implies that Muses can be used to optimise large pipelines that leverage multiple engines.
  • Textdokument
    Identifying Alternatives and Deciding Factors for a Data Mesh Architecture
    (SKILL 2022, 2022) Voß, Clara
    The data mesh was introduced in 2019 as a new type of data architecture. It promises a more democratic and scalable way of data production and consumption, while also solving data engineering problems of siloed and hyper-specialized data engineering knowledge, a growing number of dependencies within data pipelines, and the rigidness of centralized monoliths. This paper used expert interviews to identify the most significant current alternatives to the data mesh and abstract factors, with which companies can evaluate whether a data mesh can further their move to a data-driven, democratized future. The results show that the motivation, company culture, company structure, IT history and IT structure should be evaluated before implementing a data mesh. This paper is based on a bachelor thesis.
  • Konferenzbeitrag
    Integrating Access to Authority Data for Improved Interoperability of Research Data in the Digital Humanities
    (BTW 2023, 2023) Jegan, Robin; Fruth, Leon; Gradl, Tobias; Henrich, Andreas
    Authority data is used to unambiguously identify persons, organisations and places. In this paper, a means to integrate access to several providers of authority data into data curation processes is described, which facilitates disambiguation of geographic data. Combined access to general datasets, in our case the Gemeinsame Normdatei (GND), as well as highly specialized datasets, here the Memorial Archives, improves the resolution of ambiguities and particularly benefits use cases of the Digital Humanities. The integration is necessary in order to abstract from technical, syntactical and semantic heterogeneity of the providers. Operations such as querying geographic information and receiving enriched data from different data sources are facilitated. An overview of the goals of the system, related projects and authority data providers are presented, as well as details on the implementation and further steps.
  • Zeitschriftenartikel
    Integration von Daten, Anwendungen und Prozessen am Beispiel des Telekommunikationsunternehmens EWE TEL
    (Wirtschaftsinformatik: Vol. 44, No. 5, 2002) Bunjes, Bernd; Friebe, Jörg; Götze, Rainer; Harren, Arne
    The necessary specialisation in information processing in the telecommunication sector led to a wide range of software systems in the relevant software market. This can be traced back by the fact that there is a lack of capable and integrated solutions. If a telecommunication company comes to the decision to use specialised software systems, it has to face the sophisticated challenge of systems integration. This can be done at least at the levels of data, applications and processes. Illustrated by examples from the heterogeneous system environment at the telecommunication company EWE TEL, concepts for system integration and gained experiences are presented in this article. The following aspects are discussed in detail: integration of data by means of middleware, integration of applications using application server technology, and integration of processes by employing workflow management systems.
  • Zeitschriftenartikel
    ScaDS Dresden/Leipzig – A competence center for collaborative big data research
    (it - Information Technology: Vol. 60, No. 5-6, 2018) Jäkel, René; Peukert, Eric; Nagel, Wolfgang E.; Rahm, Erhard
    The efficient and intelligent handling of large, often distributed and heterogeneous data sets increasingly determines the scientific and economic competitiveness in most application areas. Mobile applications, social networks, multimedia collections, sensor networks, data intense scientific experiments, and complex simulations nowadays generate a huge data deluge. Nonetheless, processing and analyzing these data sets with innovative methods open up new opportunities for its exploitation and new insights. Nevertheless, the resulting resource requirements exceed usually the possibilities of state-of-the-art methods for the acquisition, integration, analysis and visualization of data and are summarized under the term big data. ScaDS Dresden/Leipzig, as one Germany-wide competence center for collaborative big data research, bundles efforts to realize data-intensive applications for a wide range of applications in science and industry. In this article, we present the basic concept of the competence center and give insights in some of its research topics.