QALD-10 — The 10th Challenge on Question Answering over Linked Data: Shifting from DBpedia to Wikidata as a KG for KGQA

Research output: Journal contributionsJournal articlesResearchpeer-review

Authors

  • Ricardo Usbeck
  • Xi Yan
  • Aleksandr Perevalov
  • Longquan Jiang
  • Julius Schulz
  • Angelie Kraft
  • Cedric Möller
  • Junbo Huang
  • Jan Reineke
  • Axel-Cyrille Ngonga Ngomo
  • Muhammad Saleem
  • Andreas Both
Knowledge Graph Question Answering (KGQA) has gained attention from both industry and academia over the past decade. Researchers proposed a substantial amount of benchmarking datasets with different properties, pushing the development in this field forward. Many of these benchmarks depend on Freebase, DBpedia, or Wikidata. However, KGQA benchmarks that depend on Freebase and DBpedia are gradually less studied and used, because Freebase is defunct and DBpedia lacks the structural validity of Wikidata. Therefore, research is gravitating toward Wikidata-based benchmarks. That is, new KGQA benchmarks are created on the basis of Wikidata and existing ones are migrated. We present a new, multilingual, complex KGQA benchmarking dataset as the 10th part of the Question Answering over Linked Data (QALD) benchmark series. This corpus formerly depended on DBpedia. Since QALD serves as a base for many machine-generated benchmarks, we increased the size and adjusted the benchmark to Wikidata and its ranking mechanism of properties. These measures foster novel KGQA developments by more demanding benchmarks. Creating a benchmark from scratch or migrating it from DBpedia to Wikidata is non-trivial due to the complexity of the Wikidata knowledge graph, mapping issues between different languages, and the ranking mechanism of properties using qualifiers. We present our creation strategy and the challenges we faced that will assist other researchers in their future work. Our case study, in the form of a conference challenge, is accompanied by an in-depth analysis of the created benchmark.
Original languageEnglish
JournalSemantic Web
Volume15
Issue number6
Pages (from-to)2193-2207
Number of pages14
ISSN1570-0844
DOIs
Publication statusPublished - 2023
Externally publishedYes

Recently viewed

Publications

  1. An Adaptive Resonance Regulator for an Actuator using Periodic Signals in Camless Engine Systems
  2. Class size, student performance and Tiebout bias
  3. The role of plant biodiversity in modifying the structure and functioning of higher tropic Levels in species-rich forests
  4. Explaining implementation deficits through multi-level governance in the EU's new member states
  5. Non-destructive transmissive inductive thickness sensor for IoT applications
  6. Tree diversity promotes functional dissimilarity and maintains functional richness despite species loss in predator assemblages
  7. Program for Better Riding
  8. Erratum: Formalised and non-formalised methods in resource management-knowledge and social learning in participatory processes
  9. Nonlinear control allocation applied on a QTR
  10. Socio-technical instruments in the field of Integrated Water Resources Management
  11. Nitrogen uptake by grassland communities
  12. Deep drawing of high-strength tailored blanks by using tailored tools
  13. Dynamic Semantic Web Content for Museum Guides
  14. Identification of Parameters and States in PMSMs
  15. Erroneous Examples: A Preliminary Investigation into Learning Benefits
  16. Reference wages and turnover intentions
  17. For whom are internet-based occupational mental health interventions effective? Moderators of internet-based problem-solving training outcome
  18. Towards Computer Simulations of Virtue Ethics
  19. Building trust
  20. Forgetting fire
  21. Thermodynamic description of reactions between Mg and CaO
  22. The role of supervisor support for dealing with customer verbal aggression. Differences between ethnic minority and ethnic majority workers
  23. Study of non-linear systems
  24. Using Large N Longitudinal Comparison to Explain Political Recruitment in Changing Democracies
  25. Scope of the book wastewater reuse and current challenges
  26. Deactivation and transformation products in biodegradability testing of ß-lactams amoxicillin and piperacillin
  27. Is fairness intuitive? An experiment accounting for subjective utility differences under time pressure