Auflistung nach Schlagwort "annotation variability"
1 - 1 von 1
Treffer pro Seite
Sortieroptionen
- KonferenzbeitragVariability of annotations over time: An experimental study in the dementia-related named entity recognition domain(INFORMATIK 2024, 2024) Stoev, Teodor; Suravee, Sumaiya; Yordanova, KristinaData annotation is a crucial step in various domains where Machine Learning (ML) approaches are utilized. Despite the availability of automated and semi-automated data labeling methods, manual annotation by experts remains essential for developing high-quality models in certain scenarios. This study explores how annotations can evolve over time through an experiment focused on annotating named entities and relationships within the domain of dementia and related behaviors. Two annotators labeled a task-specific text corpus on two separate occasions, one year apart. Our findings revealed an increase in both the quantity and quality of annotated entities and relationships for both annotators. Statistical tests were conducted to assess the significance of the changes in annotations. The results indicate substantial variability in annotations over time, particularly in complex domains. The paper also discusses potential reasons for these variations.