TextGraphs 2024 Shared Task on Text-Graph Representations for Knowledge Graph Question Answering

Research output: Contributions to collected editions/worksArticle in conference proceedingsResearchpeer-review

Authors

This paper describes the results of the Knowledge Graph Question Answering (KGQA) shared task that was co-located with the TextGraphs 2024 workshop. In this task, given a textual question and a list of entities with the corresponding KG subgraphs, the participating system should choose the entity that correctly answers the question. Our competition attracted thirty teams, four of which outperformed our strong ChatGPT-based zero-shot baseline. In this paper, we overview the participating systems and analyze their performance according to a large-scale automatic evaluation. To the best of our knowledge, this is the first competition aimed at the KGQA problem using the interaction between large language models (LLMs) and knowledge graphs.
Original languageEnglish
Title of host publicationProceedings of TextGraphs-17: Graph-based Methods for Natural Language Processing : Graph-Based Methods for Natural Language Processing, 62nd Annual Meeting of the Association of Computational Linguistics
EditorsDmitry Ustalov, Yanjun Gao, Alexander Pachenko, Elena Tutubalina, Irina Nikishina, Arti Ramesh, Andrey Sakhovskiy, Ricardo Usbeck, Gerald Penn, Marco Valentino
Number of pages10
Place of PublicationKerrville
PublisherAssociation for Computational Linguistics (ACL)
Publication date01.08.2024
Pages116-125
ISBN (electronic)979-8-89176-145-2
Publication statusPublished - 01.08.2024
EventTextGraphs-17: Graph-based Methods for Natural Language Processing - Bangkok, Thailand
Duration: 15.08.202415.08.2024
Conference number: 17
https://sites.google.com/view/textgraphs2024

Bibliographical note

Publisher Copyright:
© 2024 Association for Computational Linguistics.