Measuring Gender Bias in German Language Generation
Publikation: Beiträge in Sammelwerken › Aufsätze in Konferenzbänden › Forschung › begutachtet
Authors
Most existing methods to measure social bias in natural language generation are specified for English language models. In this work, we developed a German regard classifier based on a newly crowd-sourced dataset. Our model meets the test set accuracy of the original English version. With the classifier, we measured binary gender bias in two large language models. The results indicate a positive bias toward female subjects for a German version of GPT-2 and similar tendencies for GPT-3. Yet, upon qualitative analysis, we found that positive regard partly corresponds to sexist stereotypes. Our findings suggest that the regard classifier should not be used as a single measure but, instead, combined with more qualitative analyses.
| Originalsprache | Englisch |
|---|---|
| Titel | INFORMATIK 2022 - Informatik in den Naturwissenschaften |
| Herausgeber | Daniel Demmler, Daniel Krupka, Hannes Federrath |
| Anzahl der Seiten | 18 |
| Erscheinungsort | Bonn |
| Verlag | Gesellschaft für Informatik e.V. |
| Erscheinungsdatum | 2022 |
| Seiten | 1257-1274 |
| ISBN (elektronisch) | 978-3-88579-720-3 |
| DOIs | |
| Publikationsstatus | Erschienen - 2022 |
| Extern publiziert | Ja |
| Veranstaltung | 52. Jahrestagung der Gesellschaft für Informatik - INFORMATIK 2022: Informatik in den Naturwissenschaften - UHH Gebäude in der Edmund-Siemers-Allee 1, Hamburg, Deutschland Dauer: 26.09.2022 → 30.09.2022 Konferenznummer: 52 https://informatik2022.gi.de/ |
Bibliographische Notiz
Funding Information:
This work presents and extends the results of Angelie Kraft's Master thesis at Universität Hamburg and inovex GmbH. Regarding any additional research and experimentation, we acknowledge the financial support from the Federal Ministry for Economic Affairs and Energy of Germany in the project CoyPu (project number 01MK21007[G]).
Funding Information:
Acknowledgments This work presents and extends the results of Angelie Kraft’s Master thesis at Universitčt Hamburg and inovex GmbH. Regarding any additional research and experimentation, we acknowledge the financial support from the Federal Ministry for Economic Affairs and Energy of Germany in the project CoyPu (project number 01MK21007[G]).
Publisher Copyright:
© 2022 Gesellschaft fur Informatik (GI). All rights reserved.
- Wirtschaftsinformatik
- Informatik