TextGraphs 2024 Shared Task on Text-Graph Representations for Knowledge Graph Question Answering

Research output: Contributions to collected editions/worksArticle in conference proceedingsResearchpeer-review

Authors

This paper describes the results of the Knowledge Graph Question Answering (KGQA) shared task that was co-located with the TextGraphs 2024 workshop. In this task, given a textual question and a list of entities with the corresponding KG subgraphs, the participating system should choose the entity that correctly answers the question. Our competition attracted thirty teams, four of which outperformed our strong ChatGPT-based zero-shot baseline. In this paper, we overview the participating systems and analyze their performance according to a large-scale automatic evaluation. To the best of our knowledge, this is the first competition aimed at the KGQA problem using the interaction between large language models (LLMs) and knowledge graphs.
Original languageEnglish
Title of host publicationProceedings of TextGraphs-17: Graph-based Methods for Natural Language Processing : Graph-Based Methods for Natural Language Processing, 62nd Annual Meeting of the Association of Computational Linguistics
EditorsDmitry Ustalov, Yanjun Gao, Alexander Pachenko, Elena Tutubalina, Irina Nikishina, Arti Ramesh, Andrey Sakhovskiy, Ricardo Usbeck, Gerald Penn, Marco Valentino
Number of pages10
Place of PublicationKerrville
PublisherAssociation for Computational Linguistics (ACL)
Publication date01.08.2024
Pages116-125
ISBN (electronic)979-8-89176-145-2
Publication statusPublished - 01.08.2024
EventTextGraphs-17: Graph-based Methods for Natural Language Processing - Bangkok, Thailand
Duration: 15.08.202415.08.2024
Conference number: 17
https://sites.google.com/view/textgraphs2024

Bibliographical note

Publisher Copyright:
© 2024 Association for Computational Linguistics.

Recently viewed

Publications

  1. ActiveMath - a Learning Platform With Semantic Web Features
  2. Building a process layer for business applications using the blackboard pattern
  3. Model-based logistic controlling of converging material flows
  4. Gain Adaptation in Sliding Mode Control Using Model Predictive Control and Disturbance Compensation with Application to Actuators
  5. Saving (in) a common world
  6. Introduction: The representative turn in EU studies
  7. Highly Efficient IPT Transmitter Circuit Based on a Novel Enhanced Class B Amplifier for Consumer Applications
  8. Retest effects in matrix test performance
  9. How does telework modify informal workplace learning and how can supervisors provide support?
  10. The representative turn in EU studies
  11. Quality System Development at the University of Graz
  12. Reporting and Analysing the Environmental Impact of Language Models on the Example of Commonsense Question Answering with External Knowledge
  13. From teacher-centered instruction to peer tutoring in the heterogeneous international classroom
  14. Agile Portfolio Management Patterns
  15. Predicting the Individual Mood Level based on Diary Data
  16. Bed-Sharing in Couples Is Associated With Increased and Stabilized REM Sleep and Sleep-Stage Synchronization
  17. Influence of Long-Lasting Static Stretching Intervention on Functional and Morphological Parameters in the Plantar Flexors
  18. Lessons learned and challenges for environmental management in Colombia
  19. Frame-based Optimal Design
  20. Detection of significant tracer gases by means of polymer gas sensors
  21. Integrating teacher and student workspaces in a technology-enhanced mathematics lecture
  22. An Experimental Approach to the Optimization of Customer Information at the Point of Sale
  23. Performance Saga: Interview 01
  24. Erroneous examples as desirable difficulty