Logo des Repositoriums
 
Workshopbeitrag

Noise over Fear of Missing Out

Vorschaubild nicht verfügbar

Volltext URI

Dokumententyp

Text/Workshop Paper

Zusatzinformation

Datum

2021

Zeitschriftentitel

ISSN der Zeitschrift

Bandtitel

Verlag

Gesellschaft für Informatik e.V.

Zusammenfassung

Natural language processing (NLP) techniques for information extraction commonly face the challenge to extract either ‘too much’ or ‘too little’ information from text. Extracting ‘too much’ means that a lot of the relevant information is captured, but also a lot of irrelevant information or ‘Noise’ is extracted. This usually results in high ‘Recall’, but lower ‘Precision’. Extracting ‘too little’ means that all of the information that is extracted is relevant, but not everything that is relevant is extracted – it is ‘missing’ information. This usually results in high ‘Precision’ and lower ‘Recall’. In this paper we present an approach combining quantitative and qualitative measures in order to evaluate the end-users’ experience with information extraction systems in addition to standard statistical metrics and interpret a preference for the above challenge. The method is applied in a case study of legal document review. Results from the case study suggest that legal professionals prefer seeing ‘too much’ over ‘too little’ when working on an AI-assisted legal document review tasks. Discussion of these results position the involvement of User Experience (UX) as a fundamental ingredient to NLP system design and evaluation.

Beschreibung

Schleith, Johannes; Hristozova, Nina; Chechmanek, Brian; Bussey, Carolyn; Michalak, Leszek (2021): Noise over Fear of Missing Out. Mensch und Computer 2021 - Workshopband. DOI: 10.18420/muc2021-mci-ws02-290. Bonn: Gesellschaft für Informatik e.V.. MCI-WS02: UCAI 2021: Workshop on User-Centered Artificial Intelligence. Ingolstadt. 5.-8. September 2021

Zitierform

Tags