Logo des Repositoriums

Auflistung nach Schlagwort "Sentiment Annotation"

1 - 2 von 2
  • Konferenzbeitrag
    Annotator-Centered Design: Towards a Tool for Sentiment and Emotion Annotation
    (INFORMATIK 2019: 50 Jahre Gesellschaft für Informatik – Informatik für Gesellschaft (Workshop-Beiträge), 2019) Schmidt, Thomas; Jakob, Marco; Wolff, Christian
    We present a first prototype of the tool SentiAnno, which is an annotation tool specifically designed for sentiment and emotion annotation in structured written texts. To design this tool, we employ the user-centered design (UCD) framework and adapt it by focusing on the annotator, thus phrasing our development approach annotator-centered design. In iterative steps, we gather requirements and feedback from annotators as soon as possible in the development process via various usability engineering methods. We propose that this design process is especially beneficial for challenging and subjective annotation tasks like sentiment and emotion annotation of literary texts. We describe our first iterations and present results of the current prototype. We show how we were able to create functionalities facilitating the annotation process by applying this annotator-centered design approach.
  • Konferenzbeitrag
    Inter-Rater Agreement and Usability: A Comparative Evaluation of Annotation Tools for Sentiment Annotation
    (INFORMATIK 2019: 50 Jahre Gesellschaft für Informatik – Informatik für Gesellschaft (Workshop-Beiträge), 2019) Schmidt, Thomas; Winterl, Brigitte; Maul, Milena; Schark, Alina; Vlad, Andrea; Wolff, Christian
    We present the results of a comparative evaluation study of five annotation tools with 50 participants in the context of sentiment and emotion annotation of literary texts. Ten participants per tool annotated 50 speeches of the play Emilia Galotti by G. E. Lessing. We evaluate the tools via standard usability and user experience questionnaires, by measuring the time needed for the annotation, and via semi-structured interviews. Based on the results we formulate a recommendation. In addition, we discuss and compare the usability metrics and methods to develop best practices for tool selection in similar contexts. Furthermore, we also highlight the relationship between inter-rater agreement and usability metrics as well as the effect of the chosen tool on annotation behavior.