A Data Center Infrastructure Monitoring Platform Based on Storm and Trident

Abstract

Sensor data of a modern data center’s cooling and power infrastructure fulfil the character- istics of data streams and are therefore suitable for stream processing. We present a stream-based monitoring platform for data center infrastructure. It is based on multiple independent collectors, which query measurements from sensors and forward them to an Apache Kafka queue. At the platform’s core is a processing cluster based on Apache Storm and its high-level Trident API. From there, results get forwarded to one or multiple data sinks. Using our system, analytical queries can be developed using a collection of universal, generic stream operators including CORRELATE, a novel operator which combines elements from multiple streams with unique semantics. Besides the platform’s general concept, the characteristics and pitfalls of our real-world implementation are also discussed in this work.