Logo des Repositoriums

Auflistung nach Schlagwort "Apache Storm"

1 - 2 von 2
  • Konferenzbeitrag
    A Data Center Infrastructure Monitoring Platform Based on Storm and Trident
    (Datenbanksysteme für Business, Technologie und Web (BTW 2017) - Workshopband, 2017) Dreissig, Felix; Pollner, Niko
    Sensor data of a modern data center’s cooling and power infrastructure fulfil the character- istics of data streams and are therefore suitable for stream processing. We present a stream-based monitoring platform for data center infrastructure. It is based on multiple independent collectors, which query measurements from sensors and forward them to an Apache Kafka queue. At the platform’s core is a processing cluster based on Apache Storm and its high-level Trident API. From there, results get forwarded to one or multiple data sinks. Using our system, analytical queries can be developed using a collection of universal, generic stream operators including CORRELATE, a novel operator which combines elements from multiple streams with unique semantics. Besides the platform’s general concept, the characteristics and pitfalls of our real-world implementation are also discussed in this work.
  • Konferenzbeitrag
    Experiences with the Model-based Generation of Big Data Pipelines
    (Datenbanksysteme für Business, Technologie und Web (BTW 2017) - Workshopband, 2017) Eichelberger, Holger; Qin, Cui; Schmid, Klaus
    Developing Big Data applications implies a lot of schematic or complex structural tasks, which can easily lead to implementation errors and incorrect analysis results. In this paper, we present a model-based approach that supports the automatic generation of code to handle these repetitive tasks, enabling data engineers to focus on the functional aspects without being distracted by technical issues. In order to identify a solution, we analyzed different Big Data stream-processing frameworks, extracted a common graph-based model for Big Data streaming applications and de- veloped a tool to graphically design and generate such applications in a model-based fashion (in this work for Apache Storm). Here, we discuss the concepts of the approach, the tooling and, in particular, experiences with the approach based on feedback of our partners.