Logo des Repositoriums

Auflistung nach Schlagwort "Benchmarking"

1 - 10 von 23
  • Konferenzbeitrag
    Automatic validation of ICAO compliance regarding head coverings: an inclusive approach concerning religious circumstances
    (BIOSIG 2023, 2023) Carla Guerra, João S. Marcos
    This paper contributes with a dataset and an algorithm that automatically verifies the compliance with the ICAO requirements related to the use of head coverings on facial images used on machine-readable travel documents. All the methods found in the literature ignore that some coverings might be accepted because of religious or cultural reasons, and basically only look for the presence of hats/caps. Our approach specifically includes the religious cases and distinguishes the head coverings that might be considered compliant. We built a dataset composed by facial images of 500 identities to accommodate these type of accessories. That data was used to fine-tune and train a classification model based on the YOLOv8 framework and we achieved state of the art results with an accuracy of 99.1% and EER of 5.7%.
  • Zeitschriftenartikel
    Benchmarking E-Government
    (Wirtschaftsinformatik: Vol. 47, No. 5, 2005) Ostermann, Herwig; Staudinger, Roland
    E-government benchmarking reports are characterized by their multitude as well as by the dissimilarity of the approaches applied to assessing development. This article will present and compare the research methods and the main results of four selected e-government benchmarking series. Built in this remarks the authors raise the question whether these reports are able to contribute to the realization of benefits inherent in the benchmarking of e-government. It will be demonstrated, that depending on the specification of different system parameters benchmarking series show varying relevance for governments seeking to exercise certain functions. Subsequent to this functional analysis the authors will discuss whether these reports represent an appropriate foundation for governments striving for the implementation of e-government designed in a comprehensive and sustainable manner. For this purpose key elements of e-government are brought together into a holistic evaluation model. Based on this comprehensive approach it is argued that the functionality of the presented benchmarking reports remains confined to certain elements of e-government as well as to certain stakeholders’ views.
  • Konferenzbeitrag
    Benchmarking E-Government Quality - Whose Quality Are We Measuring?
    (Electronic Government and Electronic Participation - Joint Proceedings of Ongoing Research of IFIP EGOV and IFIP ePart 2016, 2013) Jansen, Arild; Ølnes, Svein
    This paper analyses the results of several years of benchmarking of public online services in Norway. We compare these data, which are showing significant differences in measured quality between small and larger municipalities, with results from a comprehensive survey measuring citizens' satisfaction with public services. Finding that these observed differences are not supported by the user survey, we have to ask: whose quality are we really measuring? Many evaluation systems rely on similar heuristic methods, e.g. the EU's eGovernment benchmark 2012 framework, while the Danish benchmarking system has a different approach. The paper argues for a multi-dimensional approach to evaluation of public websites and gives some suggestions for this.
  • Zeitschriftenartikel
    Benchmarking Energy Quantification Methods to Predict Heating Energy Performance of Residential Buildings in Germany
    (Business & Information Systems Engineering: Vol. 63, No. 3, 2021) Wenninger, Simon; Wiethe, Christian
    To achieve ambitious climate goals, it is necessary to increase the rate of purposeful retrofit measures in the building sector. As a result, Energy Performance Certificates have been designed as important evaluation and rating criterion to increase the retrofit rate in the EU and Germany. Yet, today?s most frequently used and legally required methods to quantify building energy performance show low prediction accuracy, as recent research reveals. To enhance prediction accuracy, the research community introduced data-driven methods which obtained promising results. However, there are no insights in how far Energy Quantification Methods are particularly suited for energy performance prediction. In this research article the data-driven methods Artificial Neural Network, D-vine copula quantile regression, Extreme Gradient Boosting, Random Forest, and Support Vector Regression are compared with and validated by real-world Energy Performance Certificates of German residential buildings issued by qualified auditors using the engineering method required by law. The results, tested for robustness and systematic bias, show that all data-driven methods exceed the engineering method by almost 50% in terms of prediction accuracy. In contrast to existing literature favoring Artificial Neural Networks and Support Vector Regression, all tested methods show similar prediction accuracy with marginal advantages for Extreme Gradient Boosting and Support Vector Regression in terms of prediction accuracy. Given the higher prediction accuracy of data-driven methods, it seems appropriate to revise the current legislation prescribing engineering methods. In addition, data-driven methods could support different organizations, e.g., asset management, in decision-making in order to reduce financial risk and to cut expenses.
  • Konferenzbeitrag
    Benchmarking fixed-length Fingerprint Representations across different Embedding Sizes and Sensor Types
    (BIOSIG 2023, 2023) Tim Rohwedder, Daile Osorio Roig
    Traditional minutiae-based fingerprint representations consist of a variable-length set of minutiae. This necessitates a more complex comparison causing the drawback of high computational cost in one-to-many comparison. Recently, deep neural networks have been proposed to extract fixed-length embeddings from fingerprints. In this paper, we explore to what extent fingerprint texture information contained in such embeddings can be reduced in terms of dimension, while preserving high biometric performance. This is of particular interest, since it would allow to reduce the number of operations incurred at comparisons. We also study the impact in terms of recognition performance of the fingerprint textural information for two sensor types, i.e. optical and capacitive. Furthermore, the impact of rotation and translation of fingerprint images on the extraction of fingerprint embeddings is analysed. Experimental results conducted on a publicly available database reveal an optimal embedding size of 512 feature elements for the texture-based embedding part of fixed-length fingerprint representations. In addition, differences in performance between sensor types can be perceived. The source code of all experiments presented in this paper is publicly available at https://github.com/tim-rohwedder/fixed-length-fingerprint-extractors, so our work can be fully reproduced.
  • Zeitschriftenartikel
    Benchmarking Functionalities of Domestic Service Robots Through Scientific Competitions
    (KI - Künstliche Intelligenz: Vol. 33, No. 4, 2019) Basiri, Meysam; Piazza, Enrico; Matteucci, Matteo; Lima, Pedro
    Benchmarking via carefully designed competitions makes it possible to provide a common framework for the rigorous comparison of intelligent and autonomous systems; competitions may play the role of scientific experiments while being appealing both to researchers and to the general public thus promoting critical analysis of systems outside the labs. This paper describes our approach to benchmarking domestic service robots through organizing recurrent competitions under the European Robotics League. It details the tools and benchmarks designed to evaluate the performance of robots at task and functionality levels. In particular, the functionality benchmarks for object perception and navigation are described and an overview of the new benchmarks to appear in the league is presented.
  • Konferenzbeitrag
    Benchmarking Scalability of Cloud-Native Applications
    (Software Engineering 2023, 2023) Henning, Sören; Hasselbring, Wilhelm
    This contribution has been published in the journal Empirical Software Engineering (Springer Nature) in 2022 [HH22], https://doi.org/10.1007/s10664-022-10162-1.
  • Konferenzbeitrag
    Benchmarking Scalability of Stream Processing Frameworks Deployed as Microservices in the Cloud
    (Software Engineering 2024 (SE 2024), 2024) Henning, Sören; Hasselbring, Wilhelm
  • Konferenzbeitrag
    Benchmarking the Second Generation of Intel SGX for Machine Learning Workloads
    (BTW 2023, 2023) Lutsch, Adrian; Singh, Gagandeep; Mundt, Martin; Mogk, Ragnar; Binnig, Carsten
    For domains with high data privacy and protection demands, such as health care and finance, outsourcing machine learning tasks often requires additional security measures. Trusted Execution Environments like Intel SGX are a powerful tool to achieve this additional security. Until recently, Intel SGX incurred high performance costs, mainly because it was severely limited in terms of available memory and CPUs. With the second generation of SGX, Intel alleviates these problems. Therefore, we revisit previous use cases for ML secured by SGX and show initial results of a performance study for ML workloads on SGXv2.
  • Konferenzbeitrag
    A Comparison of HPC based Quantum Computing Simulators using Quantum Volume
    (INFORMATIK 2024, 2024) Boehme, Christian; van Niekerk, Lourens; Kumar, Dhiraj; Sharma, Aasish Kumar; Meisel, Tino; Paleico, Martin Leandro
    This paper compares quantum computing simulators running on a single CPU or GPU-based HPC node using the Quantum Volume benchmark commonly proposed for comparing NISQ systems. As simulators do not suffer from noise, the metric used in the comparison is the time required to simulate a set Quantum Volume. The results are important to estimate the feasibility of proof of concept studies and debugging of quantum algorithms on HPC systems. Besides benchmarks of some commonly used simulators, this paper also offers an overview of their main features, a review of the state of quantum computing simulation and quantum computing benchmarking, and some insight into the theory of Quantum Volume.