Auflistung nach Schlagwort "benchmark"
1 - 8 von 8
Treffer pro Seite
Sortieroptionen
- KonferenzbeitragBenchmarking Function Hook Latency in Cloud-Native Environments(Softwaretechnik-Trends Band 43, Heft 4, 2023) Kahlhofer, Mario; Kern, Patrick; Henning, Sören; Rass, StefanResearchers and engineers are increasingly adopting cloud-native technologies for application development and performance evaluation. While this has improved the reproducibility of benchmarks in the cloud, the complexity of cloud-native environments makes it difficult to run benchmarks reliably. Cloud-native applications are often instrumented or altered at runtime, by dynamically patching or hooking them, which introduces a significant performance overhead. Our work discusses the benchmarking-related pitfalls of the dominant cloud-native technology, Kubernetes, and how they affect performance measurements of dy namically patched or hooked applications. We present recommendations to mitigate these risks and demonstrate how an improper experimental setup can negatively impact latency measurements.
- KonferenzbeitragBenchmarking Stream Processing Frameworks for Large Scale Data Shuffling(Softwaretechnik-Trends Band 43, Heft 4, 2023) Henning, Sören; Vogel, Adriano; Leichtfried, Michael; Ertl, Otmar; Rabiser, RickDistributed stream processing frameworks help building scalable and reliable applications that perform transformations and aggregations on continuous data streams. We outline our ongoing research on designing a new benchmark for distributed stream processing frameworks. In contrast to other benchmarks, it focuses on use cases where stream processing frameworks are mainly used for redistributing data records to perform state-local aggregations, while the actual aggregation logic is considered as black-box software components. We describe our benchmark architecture based on a real-world use case, show how we imple mented it with four state-of-the-art frameworks, and give an overview of initial experimental results.
- KonferenzbeitragBetter a Microbenchmark on a Cluster than a User at the Office: Flink Cluster Benchmarking(Softwaretechnik-Trends Band 39, Heft 3, 2019) Reichelt, David Georg; Meyer, Lars-Peter; Kühne, StefanWhen operating an Apache Flink cluster, performance problems may occur on all components of its setup. Reproducing those problems in different software or hardware components and on different nodes requires systematic experiments. We present an Apache Flink cluster benchmark set for server operators which is able to measure the performance of an Apache Flink cluster. This enables spotlighting irregularities in software or hardware behaviour.
- KonferenzbeitragComparing the Performance of Data Processing Implementations(Softwaretechnik-Trends Band 43, Heft 4, 2023) Beierlieb, Lukas; Iffländer, Lukas; Prantl, Thomas; Kounev, SamuelThis paper compares the execution speed of R, Python, and Rust implementations in the context of data processing. A real-world data processing task in the form of an aggregation of benchmark measure ment results was implemented in each language, and the execution times were measured. Rust and Python showed significantly superior performance compared to the R implementation. Further, we compared the results of different Python interpreters (the most recent versions of CPython and PyPy), also resulting in measurable variations. Finally, a study of the effectiveness of multithreading was performed.
- KonferenzbeitragExperiences from Building the Open Database Performance Ranking with benchANT(Softwaretechnik-Trends Band 43, Heft 1, 2023) Seybold, Daniel; Domaschka, JörgBenchmarking is an important method to advance database management systems (DBMS) from the industry and research perspective. Ensuring transparent and reproducible results is a key requirement to ensure the acceptance and credibility of benchmarking. To advance the research towards transparent and reproducible benchmark data, we report on building an open DBMS performance ranking with 130 benchmark configurations and ensuring comparability, transparency and reproducibility. We derive the required data on cloud, resource, DBMS and benchmark level to enable transparency and reproducibility and demonstrate the generation of such data sets with benchANT. Building upon such data, we outline future research directions for DBMS performance modelling, DBMS auto-tuning and decision support.
- KonferenzbeitragOn the Validity of Performance Antipatterns at Code Level(Softwaretechnik-Trends Band 39, Heft 4, 2019) Reichelt, David Georg; Kühne, Stefan; Hasselbring, WilhelmPerformance antipatterns at code level should be avoided to assure good application performance. Performance antipatterns avoidance is hard, since it requires up-to-date knowledge of these antipatterns. Common lists of antipatterns, like the performance rules of the static code checker PMD, only contain limited information about versions and circumstances where the performance antipatterns are valid. We close this gap by prodiving a suite of 30 performance benchmarks. Each of this benchmarks checks whether the performance antipattern is measurable in Java 6, 7, 8, 11 and 12. We find that two of the 30 performance checks are not valid in the current OpenJDK 12.
- KonferenzbeitragToward Efficient Scalability Benchmarking of Event-Driven Microservice Architectures at Large Scale(Softwaretechnik-Trends Band 40, Heft 3, 2020) Henning, Sören; Hasselbring, WilhelmOver the past years, an increase in software architectures containing microservices, which process data streams of a messaging system, can be observed. We present Theodolite, a method accompanied by an open source implementation for benchmarking the scalability of such microservices as well as their employed stream processing frameworks and deployment options. According to common scalability definitions, Theodolite provides detailed insights into how resource demands evolve with increasing load intensity. However, accurate and statistically rigorous insights come at the cost of long execution times, making it impracticable to execute benchmarks for large sets of systems under test. To overcome this limitation, we raise three research questions and propose a research agenda for executing scalability benchmarks more time-efficiently and, thus, for running scalability benchmarks at large scale.
- ZeitschriftenartikelUnderstanding the effects of temporal energy-data aggregation on clustering quality(it - Information Technology: Vol. 61, No. 2-3, 2019) Trittenbach, Holger; Bach, Jakob; Böhm, KlemensEnergy data often is available at high temporal resolution, which challenges the scalability of data-analysis methods. A common way to cope with this is to aggregate data to, say, 15-minute-interval summaries. But it often is not known how much information is lost with this, i. e., how good analysis results on aggregated data actually are. In this article, we study the effects of aggregating energy data on clustering. We propose an experimental design to compare a wide range of clustering methods found in literature. We then introduce different ways to compare clustering results obtained with different aggregation schemes. Our evaluation shows that aggregation affects the clustering quality significantly. Finally, we propose guidelines to select an aggregation scheme.