TextGraphs 2024 Shared Task on Text-Graph Representations for Knowledge Graph Question Answering

Research output: Contributions to collected editions/worksArticle in conference proceedingsResearchpeer-review

Authors

This paper describes the results of the Knowledge Graph Question Answering (KGQA) shared task that was co-located with the TextGraphs 2024 workshop. In this task, given a textual question and a list of entities with the corresponding KG subgraphs, the participating system should choose the entity that correctly answers the question. Our competition attracted thirty teams, four of which outperformed our strong ChatGPT-based zero-shot baseline. In this paper, we overview the participating systems and analyze their performance according to a large-scale automatic evaluation. To the best of our knowledge, this is the first competition aimed at the KGQA problem using the interaction between large language models (LLMs) and knowledge graphs.
Original languageEnglish
Title of host publicationProceedings of TextGraphs-17: Graph-based Methods for Natural Language Processing : Graph-Based Methods for Natural Language Processing, 62nd Annual Meeting of the Association of Computational Linguistics
EditorsDmitry Ustalov, Yanjun Gao, Alexander Pachenko, Elena Tutubalina, Irina Nikishina, Arti Ramesh, Andrey Sakhovskiy, Ricardo Usbeck, Gerald Penn, Marco Valentino
Number of pages10
Place of PublicationKerrville
PublisherAssociation for Computational Linguistics (ACL)
Publication date01.08.2024
Pages116-125
ISBN (electronic)979-8-89176-145-2
Publication statusPublished - 01.08.2024
EventTextGraphs-17: Graph-based Methods for Natural Language Processing - Bangkok, Thailand
Duration: 15.08.202415.08.2024
Conference number: 17
https://sites.google.com/view/textgraphs2024

Bibliographical note

Publisher Copyright:
© 2024 Association for Computational Linguistics.

Recently viewed

Publications

  1. A Review of Latent Variable Modeling Using R - A Step-by-Step-Guide
  2. Global temporal typing patterns in foreign language writing
  3. Efficient Order Picking Methods in Robotic Mobile Fulfillment Systems
  4. OKBQA framework towards an open collaboration for development of natural language question-answering systems over knowledge bases
  5. Machine Learning and Knowledge Discovery in Databases
  6. Finding Creativity in Predictability: Seizing Kairos in Chronos Through Temporal Work in Complex Innovation Processes
  7. Mechanical performance prediction for friction riveting joints of dissimilar materials via machine learning
  8. Intersection tests for the cointegrating rank in dependent panel data
  9. Volume of Imbalance Container Prediction using Kalman Filter and Long Short-Term Memory
  10. A guided simulated annealing search for solving the pick-up and delivery problem with time windows and capacity constraints
  11. Foundations and applications of computer based material flow networks for einvironmental management
  12. From pre-processing to advanced dynamic modeling of pupil data
  13. The Replication Database: Documenting the Replicability of Psychological Science
  14. Training effects of two different unstable shoe constructions on postural control in static and dynamic testing situations
  15. Artificial intelligence
  16. Monitoring of microbially mediated corrosion and scaling processes using redox potential measurements
  17. Mimicking and anticipating others’ actions is linked to social information processing
  18. The erosion of relational values resulting from landscape simplification
  19. Metaphors and Paradigms of the Language Animal—or—The Advantage of seeing “Time Is a Resource” as a Paradigm
  20. Construct- and criterion-related validity of the German Core Self-Evaluations Scale
  21. Facing complexity through informed simplifications
  22. Special Issue The Discourse of Redundancy Introduction
  23. “Circuits of Commons”: Exploring the Connections Between Economic Lives and the Commons
  24. Soil conditions modify species diversity effects on tree functional trait expression
  25. Modelling, explaining, enacting and getting feedback: How can the acquisition of core practices in teacher education be optimally fostered?
  26. Creep behavior of AE42 based hybrid composites
  27. Developing a Complex Portrait of Content Teaching for Multilingual Learners via Nonlinear Theoretical Understandings
  28. An Overview of Electro Hydraulic Full Variable Valve Train Systems to Reduce Emissions in Internal Combustion Engines
  29. Reciprocal Relationships Between Dispositional Optimism and Work Experiences
  30. How to support teachers to give feedback to modelling tasks effectively? Results from a teacher-training-study in the Co²CA project
  31. Introduction
  32. Visual Detection of Traffic Incident through Automatic Monitoring of Vehicle Activities
  33. Semiparametric one-step estimation of a sample selection model with endogenous covariates
  34. More than a YouTube Channel
  35. How generative drawing affects the learning process
  36. Missing links
  37. Application of design of experiments for laser shock peening process optimization
  38. On the Difficulty of Forgetting
  39. A slow-fast trait continuum at the whole community level in relation to land-use intensification
  40. Measurement in Machine Vision Editorial Paper
  41. Hacking the Classroom
  42. Knowledge Spaces of Globalization
  43. Relevance of the Basset history term for Lagrangian particle dynamics
  44. Influence of measurement errors on networks
  45. A geometric approach for the model parameter estimation in a permanent magnet synchronous motor
  46. Emotion Prediction by Facial Expressions in Human-Computer Interfaces
  47. Using latent class analysis to produce a typology of environmental concern in the UK
  48. Ablation Study of a Multimodal Gat Network on Perfect Synthetic and Real-world Data to Investigate the Influence of Language Models in Invoice Recognition
  49. Implementation of Chemometric Tools to Improve Data Mining and Prioritization in LC-HRMS for Nontarget Screening of Organic Micropollutants in Complex Water Matrixes