TextGraphs 2024 Shared Task on Text-Graph Representations for Knowledge Graph Question Answering

Research output: Contributions to collected editions/worksArticle in conference proceedingsResearchpeer-review

Authors

This paper describes the results of the Knowledge Graph Question Answering (KGQA) shared task that was co-located with the TextGraphs 2024 workshop. In this task, given a textual question and a list of entities with the corresponding KG subgraphs, the participating system should choose the entity that correctly answers the question. Our competition attracted thirty teams, four of which outperformed our strong ChatGPT-based zero-shot baseline. In this paper, we overview the participating systems and analyze their performance according to a large-scale automatic evaluation. To the best of our knowledge, this is the first competition aimed at the KGQA problem using the interaction between large language models (LLMs) and knowledge graphs.
Original languageEnglish
Title of host publicationProceedings of TextGraphs-17: Graph-based Methods for Natural Language Processing : Graph-Based Methods for Natural Language Processing, 62nd Annual Meeting of the Association of Computational Linguistics
EditorsDmitry Ustalov, Yanjun Gao, Alexander Pachenko, Elena Tutubalina, Irina Nikishina, Arti Ramesh, Andrey Sakhovskiy, Ricardo Usbeck, Gerald Penn, Marco Valentino
Number of pages10
Place of PublicationKerrville
PublisherAssociation for Computational Linguistics (ACL)
Publication date01.08.2024
Pages116-125
ISBN (electronic)979-8-89176-145-2
Publication statusPublished - 01.08.2024
EventTextGraphs-17: Graph-based Methods for Natural Language Processing - Bangkok, Thailand
Duration: 15.08.202415.08.2024
Conference number: 17
https://sites.google.com/view/textgraphs2024

Bibliographical note

Publisher Copyright:
© 2024 Association for Computational Linguistics.

Recently viewed

Publications

  1. OKBQA framework towards an open collaboration for development of natural language question-answering systems over knowledge bases
  2. Advantages and disadvantages of different text coding procedures for research and practice in a school context
  3. Interpreting Strings, Weaving Threads
  4. Foundations and applications of computer based material flow networks for einvironmental management
  5. Pressure fault recognition and compensation with an adaptive feedforward regulator in a controlled hybrid actuator within engine applications
  6. Nonlinear PD fault-tolerant control for dynamic positioning of ships with actuator constraints
  7. Metaphors and Paradigms of the Language Animal—or—The Advantage of seeing “Time Is a Resource” as a Paradigm
  8. Special Issue The Discourse of Redundancy Introduction
  9. Action Errors, Error Management, and Learning in Organizations
  10. Order parameters and energies of analytic and singular vortex lines in rotating3He-A
  11. Determinants in the online distribution of digital content
  12. Next level production networks
  13. Exploiting ConvNet diversity for flooding identification
  14. Sustainable development and learning for sustainability through a regional network project
  15. Multi-view hidden markov perceptrons
  16. Markups and Concentration in the Context of Digitization
  17. Consequences of extreme weather events for developing countries based on the example of Mongolia
  18. Development and validation of the Later Life Work Index for successful management of an aging workforce
  19. Geometric Properties on the Perfect Decoupling Disturbance Control in Manufacturing Systems
  20. Optimum parameters and rate-controlling mechanisms for hot working of extruded Mg-3Sn-1Ca alloy
  21. Web 2.0 Tasks in Action
  22. Participation in multi-level policy implementation: exploring the influence of governance culture
  23. The too-much-precision effect: When and why precise anchors backfire with experts
  24. Performance Saga: Interview 06
  25. Integrating multiple elements of environmental justice into urban blue space planning using public participation geographic information systems
  26. Elution of monomers from provisional composite materials
  27. On kites, comets, and stars. Sums of eigenvector coefficients in (molecular) graphs.
  28. Method of Artificial Vision in Guide Cane for Visually Impaired People
  29. Pragmatics broadly viewed
  30. Fishing for interpretation
  31. Art 159: Composition, procedure and voting
  32. V/2. Ethisch
  33. Alternating forms of lock-in: Publishing digital news in the path of a free content culture.
  34. Implikationen der Digitalisierung für die Organisation
  35. Introduction: Manufacturing as a challenge in Industry 4.0 process
  36. User experience and behavior concerning digital scaffolding during EFL speaking practice
  37. Pivoting the Player