Standard
Measuring Gender Bias in German Language Generation. /
Kraft, Angelie; Zorn, Hans Peter; Fecht, Pascal et al.
INFORMATIK 2022 - Informatik in den Naturwissenschaften. Hrsg. / Daniel Demmler; Daniel Krupka; Hannes Federrath. Bonn: Gesellschaft für Informatik e.V., 2022. S. 1257-1274 (Lecture Notes in Informatics (LNI), Proceedings - Series of the Gesellschaft fur Informatik (GI); Band P-326).
Publikation: Beiträge in Sammelwerken › Aufsätze in Konferenzbänden › Forschung › begutachtet
Harvard
Kraft, A, Zorn, HP, Fecht, P, Simon, J, Biemann, C
& Usbeck, R 2022,
Measuring Gender Bias in German Language Generation. in D Demmler, D Krupka & H Federrath (Hrsg.),
INFORMATIK 2022 - Informatik in den Naturwissenschaften. Lecture Notes in Informatics (LNI), Proceedings - Series of the Gesellschaft fur Informatik (GI), Bd. P-326, Gesellschaft für Informatik e.V., Bonn, S. 1257-1274, 52. Jahrestagung der Gesellschaft für Informatik - INFORMATIK 2022, Hamburg, Hamburg, Deutschland,
26.09.22.
https://doi.org/10.18420/inf2022_108
APA
Kraft, A., Zorn, H. P., Fecht, P., Simon, J., Biemann, C.
, & Usbeck, R. (2022).
Measuring Gender Bias in German Language Generation. In D. Demmler, D. Krupka, & H. Federrath (Hrsg.),
INFORMATIK 2022 - Informatik in den Naturwissenschaften (S. 1257-1274). (Lecture Notes in Informatics (LNI), Proceedings - Series of the Gesellschaft fur Informatik (GI); Band P-326). Gesellschaft für Informatik e.V..
https://doi.org/10.18420/inf2022_108
Vancouver
Kraft A, Zorn HP, Fecht P, Simon J, Biemann C
, Usbeck R.
Measuring Gender Bias in German Language Generation. in Demmler D, Krupka D, Federrath H, Hrsg., INFORMATIK 2022 - Informatik in den Naturwissenschaften. Bonn: Gesellschaft für Informatik e.V. 2022. S. 1257-1274. (Lecture Notes in Informatics (LNI), Proceedings - Series of the Gesellschaft fur Informatik (GI)). doi: 10.18420/inf2022_108
Bibtex
@inbook{c32ae1abb1bf466f85bb3380d5d01a7a,
title = "Measuring Gender Bias in German Language Generation",
abstract = "Most existing methods to measure social bias in natural language generation are specified for English language models. In this work, we developed a German regard classifier based on a newly crowd-sourced dataset. Our model meets the test set accuracy of the original English version. With the classifier, we measured binary gender bias in two large language models. The results indicate a positive bias toward female subjects for a German version of GPT-2 and similar tendencies for GPT-3. Yet, upon qualitative analysis, we found that positive regard partly corresponds to sexist stereotypes. Our findings suggest that the regard classifier should not be used as a single measure but, instead, combined with more qualitative analyses.",
keywords = "gender bias, german, gpt-2, gpt-3, natural language generation, regard, stereotypes, Business informatics, Informatics",
author = "Angelie Kraft and Zorn, {Hans Peter} and Pascal Fecht and Judith Simon and Chris Biemann and Ricardo Usbeck",
note = "This work presents and extends the results of Angelie Kraft's Master thesis at Universit{\"a}t Hamburg and inovex GmbH. Regarding any additional research and experimentation, we acknowledge the financial support from the Federal Ministry for Economic Affairs and Energy of Germany in the project CoyPu (project number 01MK21007[G]). Publisher Copyright: {\textcopyright} 2022 Gesellschaft fur Informatik (GI). All rights reserved.; 52. Jahrestagung der Gesellschaft f{\"u}r Informatik - INFORMATIK 2022 ; Conference date: 26-09-2022 Through 30-09-2022",
year = "2022",
doi = "10.18420/inf2022_108",
language = "English",
series = "Lecture Notes in Informatics (LNI), Proceedings - Series of the Gesellschaft fur Informatik (GI)",
publisher = "Gesellschaft f{\"u}r Informatik e.V.",
pages = "1257--1274",
editor = "Daniel Demmler and Daniel Krupka and Hannes Federrath",
booktitle = "INFORMATIK 2022 - Informatik in den Naturwissenschaften",
address = "Germany",
url = "https://informatik2022.gi.de/",
}
RIS
TY - CHAP
T1 - Measuring Gender Bias in German Language Generation
AU - Kraft, Angelie
AU - Zorn, Hans Peter
AU - Fecht, Pascal
AU - Simon, Judith
AU - Biemann, Chris
AU - Usbeck, Ricardo
N1 - Conference code: 52
PY - 2022
Y1 - 2022
N2 - Most existing methods to measure social bias in natural language generation are specified for English language models. In this work, we developed a German regard classifier based on a newly crowd-sourced dataset. Our model meets the test set accuracy of the original English version. With the classifier, we measured binary gender bias in two large language models. The results indicate a positive bias toward female subjects for a German version of GPT-2 and similar tendencies for GPT-3. Yet, upon qualitative analysis, we found that positive regard partly corresponds to sexist stereotypes. Our findings suggest that the regard classifier should not be used as a single measure but, instead, combined with more qualitative analyses.
AB - Most existing methods to measure social bias in natural language generation are specified for English language models. In this work, we developed a German regard classifier based on a newly crowd-sourced dataset. Our model meets the test set accuracy of the original English version. With the classifier, we measured binary gender bias in two large language models. The results indicate a positive bias toward female subjects for a German version of GPT-2 and similar tendencies for GPT-3. Yet, upon qualitative analysis, we found that positive regard partly corresponds to sexist stereotypes. Our findings suggest that the regard classifier should not be used as a single measure but, instead, combined with more qualitative analyses.
KW - gender bias
KW - german
KW - gpt-2
KW - gpt-3
KW - natural language generation
KW - regard
KW - stereotypes
KW - Business informatics
KW - Informatics
UR - http://www.scopus.com/inward/record.url?scp=85139747989&partnerID=8YFLogxK
U2 - 10.18420/inf2022_108
DO - 10.18420/inf2022_108
M3 - Article in conference proceedings
AN - SCOPUS:85139747989
T3 - Lecture Notes in Informatics (LNI), Proceedings - Series of the Gesellschaft fur Informatik (GI)
SP - 1257
EP - 1274
BT - INFORMATIK 2022 - Informatik in den Naturwissenschaften
A2 - Demmler, Daniel
A2 - Krupka, Daniel
A2 - Federrath, Hannes
PB - Gesellschaft für Informatik e.V.
CY - Bonn
T2 - 52. Jahrestagung der Gesellschaft für Informatik - INFORMATIK 2022
Y2 - 26 September 2022 through 30 September 2022
ER -
