Logo des Repositoriums

Auflistung nach Schlagwort "Replicability"

1 - 4 von 4
  • Konferenzbeitrag
    Community Expectations for Research Artifacts and Evaluation Processes
    (Software Engineering 2023, 2023) Hermann, Ben; Winter, Stefan; Siegmund, Janet
    Artifact evaluation has been introduced into the software engineering and programming languages research community with a pilot at ESEC/FSE 2011 and has since then enjoyed a healthy adoption throughout the conference landscape. We conducted a survey including all members of artifact evaluation committees of major conferences in the software engineering and programming language field from 2011 to 2019 and compared the answers to expectations set by calls for artifacts and reviewing guidelines. While we find that some expectations exceed the ones expressed in calls and reviewing guidelines, there is no consensus on a quality threshold for artifacts in general. We observe very specific quality expectations for specific artifact types for review and later usage, but also a lack of their communication in calls. We also find problematic inconsistencies in the terminology used to express artifact evaluation’s most important purpose. We derive several actionable suggestions which can help to mature artifact evaluation in the inspected community and also to aid its introduction into other communities in computer science.
  • Zeitschriftenartikel
    Guest editorial: Information security methodology and replication studies
    (it - Information Technology: Vol. 64, No. 1-2, 2022) Wendzel, Steffen; Caviglione, Luca; Mileva, Aleksandra; Lalande, Jean-Francois; Mazurczyk, Wojciech
    This special issue presents five articles that address the topic of replicability and scientific methodology in information security research, featuring two extended articles from the 2021 International Workshop on Information Security Methodology and Replication Studies (IWSMR). This special issue also comprises two distinguished dissertations.
  • Zeitschriftenartikel
    Mapping platforms into a new open science model for machine learning
    (it - Information Technology: Vol. 61, No. 4, 2019) Weißgerber, Thomas; Granitzer, Michael
    Data-centric disciplines like machine learning and data science have become major research areas within computer science and beyond. However, the development of research processes and tools did not keep pace with the rapid advancement of the disciplines, resulting in several insufficiently tackled challenges to attain reproducibility, replicability, and comparability of achieved results. In this discussion paper, we review existing tools, platforms and standardization efforts for addressing these challenges. As a common ground for our analysis, we develop an open science centred process model for machine learning research, which combines openness and transparency with the core processes of machine learning and data science. Based on the features of over 40 tools, platforms and standards, we list the, in our opinion, 11 most central platforms for the research process in this paper. We conclude that most platforms cover only parts of the requirements for overcoming the identified challenges.
  • Zeitschriftenartikel
    Quantitative visual computing
    (it - Information Technology: Vol. 64, No. 4-5, 2022) Schreiber; Falk; Weiskopf, Daniel
    This special issue presents five articles that address the topic of replicability and scientific methodology in information security research, featuring two extended articles from the 2021 International Workshop on Information Security Methodology and Replication Studies (IWSMR). This special issue also comprises two distinguished dissertations.