Logo des Repositoriums
 

Variability of annotations over time: An experimental study in the dementia-related named entity recognition domain

Kurzanzeige
dc.contributor.authorStoev, Teodor
dc.contributor.authorSuravee, Sumaiya
dc.contributor.authorYordanova, Kristina
dc.contributor.editorKlein, Maike
dc.contributor.editorKrupka, Daniel
dc.contributor.editorWinter, Cornelia
dc.contributor.editorGergeleit, Martin
dc.contributor.editorMartin, Ludger
dc.date.accessioned2024-10-21T18:24:25Z
dc.date.available2024-10-21T18:24:25Z
dc.date.issued2024
dc.description.abstractData annotation is a crucial step in various domains where Machine Learning (ML) approaches are utilized. Despite the availability of automated and semi-automated data labeling methods, manual annotation by experts remains essential for developing high-quality models in certain scenarios. This study explores how annotations can evolve over time through an experiment focused on annotating named entities and relationships within the domain of dementia and related behaviors. Two annotators labeled a task-specific text corpus on two separate occasions, one year apart. Our findings revealed an increase in both the quantity and quality of annotated entities and relationships for both annotators. Statistical tests were conducted to assess the significance of the changes in annotations. The results indicate substantial variability in annotations over time, particularly in complex domains. The paper also discusses potential reasons for these variations.en
dc.identifier.doi10.18420/inf2024_35
dc.identifier.isbn978-3-88579-746-3
dc.identifier.pissn1617-5468
dc.identifier.urihttps://dl.gi.de/handle/20.500.12116/45194
dc.language.isoen
dc.publisherGesellschaft für Informatik e.V.
dc.relation.ispartofINFORMATIK 2024
dc.relation.ispartofseriesLecture Notes in Informatics (LNI) - Proceedings, Volume P-352
dc.subjectdata annotation
dc.subjectannotation evolution
dc.subjectannotators
dc.subjectannotation variability
dc.subjectdata labeling
dc.titleVariability of annotations over time: An experimental study in the dementia-related named entity recognition domainen
dc.typeText/Conference Paper
gi.citation.endPage486
gi.citation.publisherPlaceBonn
gi.citation.startPage473
gi.conference.date24.-26. September 2024
gi.conference.locationWiesbaden
gi.conference.sessiontitle8th International Workshop on Annotation of useR Data for UbiquitOUs Systems

Dateien

Originalbündel
1 - 1 von 1
Vorschaubild
Name:
Stoev_et_al_Variability_of_annotations.pdf
Größe:
405.69 KB
Format:
Adobe Portable Document Format
Herunterladen

Sammlungen

P352 - INFORMATIK 2024 - Lock in or log out? Wie digitale Souveränität gelingt