8th challenge on question answering over linked data (QALD-8)

Research output: Contributions to collected editions/worksArticle in conference proceedingsResearchpeer-review

Authors

  • Ricardo Usbeck
  • Axel Cyrille Ngonga Ngomo
  • Felix Conrads
  • Michael Röder
  • Giulio Napolitano

The QALD-8 challenge focused on the successful and long running multilingual QA task. For the first time, the participating teams were required to provide webservices of their systems to participate in the challenge, which will in turn support comparable research in the future. In this challenge, we also changed the underlying evaluation platform to account for the need for comparable experiments via webservices in contrast to former XML/JSON file submissions. This increased the entrance requirements for participating teams but ensures long term comparability of the system performance and a fair and open challenge. In the future, we will further simplify the participation process and offer leaderboards prior to the actual challenge to allow participants to see their performance beforehand. After feedback from the authors, we will likely add new key performance indicators for the capability of a system to know which questions it cannot answer and take confidence scores for answers into account. Moreover, we will remove most of the curve ball questions to reflect the original character of the QALD challenge, which provides a clean and linguistically challenging benchmark.

Original languageEnglish
Title of host publicationSemDeep-4_NLIWOD-4 2018 : Joint Proceedings of ISWC 2018 Workshops SemDeep-4 and NLIWOD-4
EditorsKey-Sun Choi, Luis Espinosa Anke, Thierry Decklerck, Dagmar Gromann, Jin-Dong Kim, Axel-Cyrille Ngonga Ngomo, Muhammad Saleem, Ricardo Usbeck
Number of pages7
Volume2241
PublisherSun Site Central Europe (RWTH Aachen University)
Publication date10.2018
Pages51-57
Publication statusPublished - 10.2018
Externally publishedYes
EventJoint 4th Workshop on Semantic Deep Learning: Natural Language Interfaces for the Web of Data and 9th Question Answering over Linked Data Challenge, SemDeep-4_NLIWOD-4 2018: co-located with the 17th International Semantic Web Conference - Monterey, United States
Duration: 08.10.201809.10.2018
https://ceur-ws.org/Vol-2241/
https://ceur-ws.org/Vol-2241/JointPreface.pdf

Bibliographical note

Publisher Copyright:
© 2018 CEUR-WS. All Rights Reserved.

Recently viewed

Researchers

  1. Marcus Erben

Activities

  1. The golden age of software architecture better named the middle age of software architecture - Some provocative thoughts
  2. EXIT
  3. From Magic to Systemics. Heinz von Foerster and the Reenchantment of Science
  4. Co-supervisor of the dissertation "Diversity and functions of plant-insect interactions along a forest retention gradient"
  5. ICSB World Conference - ICSB 2007
  6. Towards decent platform work
  7. Thomas Weise
  8. Workshop on Family Migration Processes in a Comparative Perspective - 2018
  9. 27th International Conference on System Theory, Control and Computing - ICSTCC 2023
  10. Monkey Business: Who Pulls the Strings? 2013
  11. Concurrent Adaptive Tests for Formative Assessments in School Classes
  12. Empirical Research Methods on Legitimacy: Repertory Grid as the Interface between „Measuring“ and „Evaluating“
  13. (De)composing Public Value: New Evidence for Basic Structures
  14. Speaking about vision, talking in the name of so much more: A methodological framework for ventriloquial analyses in organization studies
  15. Explaining Healthcare System Change
  16. Knowledge Space(s) of Globalization – Musealizing Things, People and Spaces of Global Trade
  17. Digital Games Lab Lecture Series - 2018
  18. How does tree sapling diversity influence browsing intensity by deer across spatial scales?
  19. Vortrag: Teaching football skills with digitally supported learning processes in primary school
  20. Going Green: Digital project work as a transdisciplinary and transcultural task in the foreign language and STEM classrooms
  21. Trajectory-based Lagrangian approaches for the extraction and characterization of coherent structures in turbulent convection
  22. International Conference of Computational Methods in Engineering Science - Chair of Session III

Publications

  1. Operations Systems of Container Terminals
  2. Entity linking in 40 languages using MAG
  3. Object-Oriented Construction Handbook
  4. Recognition and approach responses toward threatening objects
  5. Conjunctive cohesion in English language EU documents - A corpus-based analysis and its implications
  6. Towards a spatial understanding of identity play
  7. On the distinctiveness of tags in collaborative tagging systems
  8. Developing a sustainable platform for entity annotation benchmarks
  9. NNARX networks on didactic level system identification
  10. Preventive Diagnostics for cardiovascular diseases based on probabilistic methods and description logic
  11. Highly Efficient IPT Transmitter Circuit Based on a Novel Enhanced Class B Amplifier for Consumer Applications
  12. Retest effects in matrix test performance
  13. Implementation of formative assessment
  14. TRY plant trait database – enhanced coverage and open access
  15. Modern Baselines for SPARQL Semantic Parsing
  16. Performance of methods to select landscape metrics for modelling species richness
  17. Earnings Less Risk-Free Interest Charge (ERIC) and Stock Returns—A Value-Based Management Perspective on ERIC’s Relative and Incremental Information Content
  18. Automatic three-dimensional geometry and mesh generation of periodic representative volume elements for matrix-inclusion composites
  19. An empirically grounded ontology for analyzing IT-based interventions in business ecosystems
  20. Reducing mean tardiness in a flexible job shop containing AGVs with optimized combinations of sequencing and routing rules