Noise over Fear of Missing Out
| dc.contributor.author | Schleith, Johannes | |
| dc.contributor.author | Hristozova, Nina | |
| dc.contributor.author | Chechmanek, Brian | |
| dc.contributor.author | Bussey, Carolyn | |
| dc.contributor.author | Michalak, Leszek | |
| dc.contributor.editor | Wienrich, Carolin | |
| dc.contributor.editor | Wintersberger, Philipp | |
| dc.contributor.editor | Weyers, Benjamin | |
| dc.date.accessioned | 2021-09-05T18:56:36Z | |
| dc.date.available | 2021-09-05T18:56:36Z | |
| dc.date.issued | 2021 | |
| dc.description.abstract | Natural language processing (NLP) techniques for information extraction commonly face the challenge to extract either ‘too much’ or ‘too little’ information from text. Extracting ‘too much’ means that a lot of the relevant information is captured, but also a lot of irrelevant information or ‘Noise’ is extracted. This usually results in high ‘Recall’, but lower ‘Precision’. Extracting ‘too little’ means that all of the information that is extracted is relevant, but not everything that is relevant is extracted – it is ‘missing’ information. This usually results in high ‘Precision’ and lower ‘Recall’. In this paper we present an approach combining quantitative and qualitative measures in order to evaluate the end-users’ experience with information extraction systems in addition to standard statistical metrics and interpret a preference for the above challenge. The method is applied in a case study of legal document review. Results from the case study suggest that legal professionals prefer seeing ‘too much’ over ‘too little’ when working on an AI-assisted legal document review tasks. Discussion of these results position the involvement of User Experience (UX) as a fundamental ingredient to NLP system design and evaluation. | en |
| dc.identifier.doi | 10.18420/muc2021-mci-ws02-290 | |
| dc.identifier.uri | https://dl.gi.de/handle/20.500.12116/37374 | |
| dc.language.iso | en | |
| dc.publisher | Gesellschaft für Informatik e.V. | |
| dc.relation.ispartof | Mensch und Computer 2021 - Workshopband | |
| dc.relation.ispartofseries | Mensch und Computer | |
| dc.subject | User centric evaluation of AI-based systems | |
| dc.subject | Precision | |
| dc.subject | Recall | |
| dc.subject | Task Support | |
| dc.title | Noise over Fear of Missing Out | en |
| dc.type | Text/Workshop Paper | |
| gi.citation.publisherPlace | Bonn | |
| gi.conference.date | 5.-8. September 2021 | |
| gi.conference.location | Ingolstadt | |
| gi.conference.sessiontitle | MCI-WS02: UCAI 2021: Workshop on User-Centered Artificial Intelligence | |
| gi.document.quality | digidoc |
Dateien
Originalbündel
1 - 1 von 1
Vorschaubild nicht verfügbar
- Name:
- Contribution_290__a.pdf
- Größe:
- 448.6 KB
- Format:
- Adobe Portable Document Format