Knowledge-Enhanced Language Models Are Not Bias-Proof: Situated Knowledge and Epistemic Injustice in AI

Publikation: Beiträge in SammelwerkenAufsätze in KonferenzbändenForschungbegutachtet

Authors

The factual inaccuracies ("hallucinations") of large language models have recently inspired more research on knowledge-enhanced language modeling approaches. These are often assumed to enhance the overall trustworthiness and objectivity of language models. Meanwhile, the issue of bias is usually only mentioned as a limitation of statistical representations. This dissociation of knowledge-enhancement and bias is in line with previous research on AI engineers' assumptions about knowledge, which indicate that knowledge is commonly understood as objective and value-neutral by this community. We argue that claims and practices by actors of the field still reflect this underlying conception of knowledge. We contrast this assumption with literature from social and, in particular, feminist epistemology, which argues that the idea of a universal disembodied knower is blind to the reality of knowledge practices and seriously challenges claims of "objective"or "neutral"knowledge. Knowledge enhancement techniques commonly use Wikidata and Wikipedia as their sources for knowledge, due to their large scales, public accessibility, and assumed trustworthiness. In this work, they serve as a case study for the influence of the social setting and the identity of knowers on epistemic processes. Indeed, the communities behind Wikidata and Wikipedia are known to be male-dominated and many instances of hostile behavior have been reported in the past decade. In effect, the contents of these knowledge bases are highly biased. It is therefore doubtful that these knowledge bases would contribute to bias reduction. In fact, our empirical evaluations of RoBERTa, KEPLER, and CoLAKE, demonstrate that knowledge enhancement may not live up to the hopes of increased objectivity. In our study, the average probability for stereotypical associations was preserved on two out of three metrics and performance-related gender gaps on knowledge-driven task were also preserved. We build on these results and critical literature to argue that the label of "knowledge"and the commonly held beliefs about it can obscure the harm that is still done to marginalized groups. Knowledge enhancement is at risk of perpetuating epistemic injustice, and AI engineers' understanding of knowledge as objective per se conceals this injustice. Finally, to get closer to trustworthy language models, we need to rethink knowledge in AI and aim for an agenda of diversification and scrutiny from outgroup members.

OriginalspracheEnglisch
Titel2024 ACM Conference on Fairness, Accountability, and Transparency, FAccT 2024
Anzahl der Seiten13
VerlagAssociation for Computing Machinery, Inc
Erscheinungsdatum03.06.2024
Seiten1433-1445
ISBN (Print)9798400704505
ISBN (elektronisch)979-8-4007-0450-5
DOIs
PublikationsstatusErschienen - 03.06.2024
VeranstaltungACM Conference on Fairness, Accountability, and Transparency - FAccT 2024 - Rio de Janeiro, Brasilien
Dauer: 03.06.202406.06.2024
https://facctconference.org/2024/

Bibliographische Notiz

Publisher Copyright:
© 2024 Owner/Author.

DOI

Zuletzt angesehen

Publikationen

  1. Determination of the construction and the material identity values of outside building components with the help of in-situ measuring procedures and FEM-simulation calculations
  2. Repeat Receipts: A device for generating visible data in market research focus groups
  3. The Network Dynamics of Movements
  4. Performance concepts and performance theory
  5. Multilayer neural networks
  6. User Authentication via Multifaceted Mouse Movements and Outlier Exposure
  7. Joint Proceedings of Scholarly QALD 2023 and SemREC 2023 co-located with 22nd International Semantic Web Conference ISWC 2023
  8. An Overview of Electro Hydraulic Full Variable Valve Train Systems to Reduce Emissions in Internal Combustion Engines
  9. The measurement time required for determining total NH3 losses after field application of slurries by trail hoses
  10. Recontextualizing context
  11. Manual for Analysis of Soils and Related Materials
  12. Using a decoupling technique to identify the magnetic flux in a permanent magnet synchronous motor
  13. Analysis of the mechanical properties of an arc-sprayed WC-FeCSiMn coating
  14. Sensorless Control of AC Motor Drives with Adaptive Extended Kalman Filter
  15. Velocity-free friction compensation for motion systems with actuator constraint
  16. A Robust Approximated Derivative Action of a PID Regulator to be Applied in a Permanent Magnet Synchronous Motor Control
  17. Rebound Effects in Methods of Artificial Intelligence
  18. In situ synchrotron radiation diffraction during solidification of Mgl5Gd
  19. Systematic risk behavior in cyclical industries
  20. The importance of product lifetime labelling for purchase decisions
  21. Der FFB-Server mit Microsoft Windows Server 2003
  22. The influence of vertical integration and property rights on network access charges in the German electricity market