TextGraphs 2024 Shared Task on Text-Graph Representations for Knowledge Graph Question Answering

Research output: Contributions to collected editions/worksArticle in conference proceedingsResearchpeer-review

Authors

This paper describes the results of the Knowledge Graph Question Answering (KGQA) shared task that was co-located with the TextGraphs 2024 workshop. In this task, given a textual question and a list of entities with the corresponding KG subgraphs, the participating system should choose the entity that correctly answers the question. Our competition attracted thirty teams, four of which outperformed our strong ChatGPT-based zero-shot baseline. In this paper, we overview the participating systems and analyze their performance according to a large-scale automatic evaluation. To the best of our knowledge, this is the first competition aimed at the KGQA problem using the interaction between large language models (LLMs) and knowledge graphs.
Original languageEnglish
Title of host publicationProceedings of TextGraphs-17: Graph-based Methods for Natural Language Processing : Graph-Based Methods for Natural Language Processing, 62nd Annual Meeting of the Association of Computational Linguistics
EditorsDmitry Ustalov, Yanjun Gao, Alexander Pachenko, Elena Tutubalina, Irina Nikishina, Arti Ramesh, Andrey Sakhovskiy, Ricardo Usbeck, Gerald Penn, Marco Valentino
Number of pages10
Place of PublicationKerrville
PublisherAssociation for Computational Linguistics (ACL)
Publication date01.08.2024
Pages116-125
ISBN (electronic)979-8-89176-145-2
Publication statusPublished - 01.08.2024
EventTextGraphs-17: Graph-based Methods for Natural Language Processing - Bangkok, Thailand
Duration: 15.08.202415.08.2024
Conference number: 17
https://sites.google.com/view/textgraphs2024

Bibliographical note

Publisher Copyright:
© 2024 Association for Computational Linguistics.

Recently viewed

Publications

  1. Q-Adaptive Control of the nonlinear dynamics of the cantilever-sample system of an Atomic Force Microscope
  2. XOperator - Interconnecting the semantic web and instant messaging networks
  3. Comparison of Odor Thresholds obtained by a Three Alternative Choice Procedure and by the Method of Limits
  4. Foundations and applications of computer based material flow networks for einvironmental management
  5. Teaching methods for modelling problems and students’ task-specific enjoyment, value, interest and self-efficacy expectations
  6. Robust feedback linearization using an adaptive PD regulator for a sensorless control of a throttle valve
  7. How, when and why do negotiators use reference points?
  8. Does thinking-aloud affect learning, visual information processing and cognitive load when learning with seductive details as expected from self-regulation perspective?
  9. Are Acute Effects of Foam-Rolling Attributed to Dynamic Warm Up Effects? A Comparative Study
  10. Modelling and Optimization of Commuter Flows as Queuing System Considering Customer and Environmental Costs
  11. Repeat Receipts: A device for generating visible data in market research focus groups
  12. Making mutual learning tangible
  13. Efficacy of a Web-Based Intervention With Mobile Phone Support in Treating Depressive Symptoms in Adults With Type 1 and Type 2 Diabetes
  14. Unveiling local knowledge
  15. Variational pragmatics in the foreign language classroom
  16. Forging of Mg–3Sn–2Ca–0.4Al Alloy Assisted by Its Processing Map and Validation Through Analytical Modeling
  17. Creep behavior of AE42 based hybrid composites
  18. Simulation and optimization of material and energy flow systems
  19. How to support teachers to give feedback to modelling tasks effectively? Results from a teacher-training-study in the Co²CA project
  20. Geodesign as a boundary management process
  21. Nonlinear anisotropic boundary value problems – regularity results and multiscale discretizations
  22. Practical Formalist
  23. Adaptive Item Selection Under Matroid Constraints
  24. Performance incentives in activity-based management
  25. Design, Modeling and Control of an Over-actuated Hexacopter Tilt-Rotor
  26. Using Long-Duration Static Stretch Training to Counteract Strength and Flexibility Deficits in Moderately Trained Participants
  27. Decisions And Characteristics During The Development Process Of A Software Demonstrator For Data Analysis In Production Logistics
  28. One tool to rule? – A field experimental longitudinal study on the costs and benefits of mobile device usage in public agencies
  29. Discrete Lyapunov Controllers for an Actuator in Camless Engines
  30. Proxy Indicators for the Quality of Open-domain Dialogues
  31. Tree diversity alters the structure of a tri-trophic network in a biodiversity experiment
  32. Time Use Research and Time Use Data
  33. Exploring the dark and unexpected sides of digitalization
  34. Using Reading Strategy Training to Foster Students´ Mathematical Modelling Competencies