Textdokument

Measuring Gender Bias in German Language Generation

Vorschaubild nicht verfügbar
Volltext URI
Dokumententyp
Datum
2022
Zeitschriftentitel
ISSN der Zeitschrift
Bandtitel
Quelle
INFORMATIK 2022
Trustworthy AI in Science and Society
Verlag
Gesellschaft für Informatik, Bonn
Zusammenfassung
Most existing methods to measure social bias in natural language generation are specified for English language models. In this work, we developed a German regard classifier based on a newly crowd-sourced dataset. Our model meets the test set accuracy of the original English version. With the classifier, we measured binary gender bias in two large language models. The results indicate a positive bias toward female subjects for a German version of GPT-2 and similar tendencies for GPT-3. Yet, upon qualitative analysis, we found that positive regard partly corresponds to sexist stereotypes. Our findings suggest that the regard classifier should not be used as a single measure but, instead, combined with more qualitative analyses.
Beschreibung
Kraft,Angelie; Zorn,Hans-Peter; Fecht,Pascal; Simon,Judith; Biemann,Chris; Usbeck,Ricardo (2022): Measuring Gender Bias in German Language Generation. INFORMATIK 2022. DOI: 10.18420/inf2022_108. Gesellschaft für Informatik, Bonn. PISSN: 1617-5468. ISBN: 978-3-88579-720-3. pp. 1257-1274. Trustworthy AI in Science and Society. Hamburg. 26.-30. September 2022
Zitierform
Tags