Dissertation
Interactive tools for reproducible science: understanding, supporting, and motivating reproducible science practices
Vorschaubild nicht verfügbar
Volltext URI
Dokumententyp
Text/Dissertation
Zusatzinformation
Datum
2020
Autor:innen
Zeitschriftentitel
ISSN der Zeitschrift
Bandtitel
Quelle
Verlag
Ludwig-Maximilians-Universität München, Fakultät für Mathematik, Informatik und Statistik
Zusammenfassung
Reproducibility should be a cornerstone of science. It plays an essential role in research validation and reuse. In recent years, the scientific community and the general public became increasingly aware of the reproducibility crisis, i.e. the wide-spread inability of researchers to reproduce published work, including their own. The reproducibility crisis has been identified in most branches of data-driven science. The effort required to document, clean, preserve, and share experimental resources has been described as one of the core contributors to this irreproducibility challenge. Documentation, preservation, and sharing are key reproducible research practices that are of little perceived value for scientists, as they fall outside the traditional academic reputation economy that is focused on novelty-driven scientific contributions.
Scientific research is increasingly focused on the creation, observation, processing, and analysis of large data volumes. On one hand, this transition towards computational and data-intensive science poses new challenges for research reproducibility and reuse. On the other hand, increased availability and advances in computation and web technologies offer new opportunities to address the reproducibility crisis. A prominent example is the World Wide Web (WWW), which was developed in response to researchers’ needs to quickly share research data and findings with the scientific community. The WWW was invented at the European Organization for Nuclear Research (CERN). CERN is a key laboratory in High Energy Physics (HEP), one of the most data-intensive scientific domains. This thesis reports on research connected in the context of CAP, a Research Data Management (RDM) service tailored to CERN's major experiments. We use this scientific environment to study the role and requirements of interactive tools in facilitating reproducible research.
In this thesis, we build a wider understanding of researchers' interactions with tools that support research documentation, preservation, and sharing. From an HCI perspective the following aspects are fundamental: (1) Characterize and map requirements and practices around research preservation and reuse. (2) Understand the wider role and impact of RDM tools in scientific workflows. (3) Design tools and interactions that promote, motivate, and acknowledge reproducible research practices.
Research reported in this thesis represents the first systematic application of HCI methods in the study and design of interactive tools for reproducible science. We have built an empirical understanding of reproducible research practices and the role of supportive tools through research in HEP and across a variety of scientific fields. We designed prototypes and implemented services that aim to create rewarding and motivating interactions. We conducted mixed-method evaluations to assess the UX of the designs, in particular related to usefulness, suitability, and persuasiveness. We report on four empirical studies in which 42 researchers and data managers participated.
In the first interview study, we asked HEP data analysts about RDM practices and invited them to explore and discuss CAP. Our findings show that tailored preservation services allow for introducing and promoting meaningful rewards and incentives that benefit contributors in their research work. Here, we introduce the term secondary usage forms of RDM tools. While not part of the core mission of the tools, secondary usage forms motivate contributions through meaningful rewards. We extended this research through a cross-domain interview study with data analysts and data stewards from a diverse set of scientific fields. Based on the findings of this cross-domain study, we contribute a Stage-Based Model of Personal RDM Commitment Evolution that explains how and why scientists commit to open and reproducible science.
To address the motivation challenge, we explored if and how gamification can motivate contributions and promote reproducible research practices. To this end, we designed two prototypes of a gamified preservation service that was inspired by CAP. Each gamification prototype makes use of different underlying mechanisms. HEP researchers found both implementations valuable, enjoyable, suitable, and persuasive. The gamification layer improves visibility of scientists and research work and facilitates content navigation and discovery. Based on these findings, we implemented six tailored science badges in CAP in our second gamification study. The badges promote and reward high-quality documentation and special uses of preserved research. Findings from our evaluation with HEP researchers show that tailored science badges enable novel forms of research repository navigation and content discovery that benefit users and contributors. We discuss how the use of tailored science badges as an incentivizing element paves new ways for interaction with research repositories.
Finally, we describe the role of HCI in supporting reproducible research practices. We stress that tailored RDM tools can improve content navigation and discovery, which is key in the design of secondary usage forms. Moreover, we argue that incentivizing elements like gamification may not only motivate contributions, but further promote secondary uses and enable new forms of interaction with preserved research. Based on our empirical research, we describe the roles of both HCI scholars and practitioners in building interactive tools for reproducible science. Finally, we outline our vision to transform computational and data-driven research preservation through ubiquitous preservation strategies that integrate into research workflows and make use of automated knowledge recording.
In conclusion, this thesis advocates the unique role of HCI in supporting, motivating, and transforming reproducible research practices through the design of tools that enable effective RDM. We present practices around research preservation and reuse in HEP and beyond. Our research paves new ways for interaction with RDM tools that support and motivate reproducible science.