8th challenge on question answering over linked data (QALD-8)

Research output: Contributions to collected editions/worksArticle in conference proceedingsResearchpeer-review

Authors

  • Ricardo Usbeck
  • Axel Cyrille Ngonga Ngomo
  • Felix Conrads
  • Michael Röder
  • Giulio Napolitano

The QALD-8 challenge focused on the successful and long running multilingual QA task. For the first time, the participating teams were required to provide webservices of their systems to participate in the challenge, which will in turn support comparable research in the future. In this challenge, we also changed the underlying evaluation platform to account for the need for comparable experiments via webservices in contrast to former XML/JSON file submissions. This increased the entrance requirements for participating teams but ensures long term comparability of the system performance and a fair and open challenge. In the future, we will further simplify the participation process and offer leaderboards prior to the actual challenge to allow participants to see their performance beforehand. After feedback from the authors, we will likely add new key performance indicators for the capability of a system to know which questions it cannot answer and take confidence scores for answers into account. Moreover, we will remove most of the curve ball questions to reflect the original character of the QALD challenge, which provides a clean and linguistically challenging benchmark.

Original languageEnglish
Title of host publicationSemDeep-4_NLIWOD-4 2018 : Joint Proceedings of ISWC 2018 Workshops SemDeep-4 and NLIWOD-4
EditorsKey-Sun Choi, Luis Espinosa Anke, Thierry Decklerck, Dagmar Gromann, Jin-Dong Kim, Axel-Cyrille Ngonga Ngomo, Muhammad Saleem, Ricardo Usbeck
Number of pages7
Volume2241
PublisherSun Site Central Europe (RWTH Aachen University)
Publication date10.2018
Pages51-57
Publication statusPublished - 10.2018
Externally publishedYes
EventJoint 4th Workshop on Semantic Deep Learning: Natural Language Interfaces for the Web of Data and 9th Question Answering over Linked Data Challenge, SemDeep-4_NLIWOD-4 2018: co-located with the 17th International Semantic Web Conference - Monterey, United States
Duration: 08.10.201809.10.2018
https://ceur-ws.org/Vol-2241/
https://ceur-ws.org/Vol-2241/JointPreface.pdf

Bibliographical note

Publisher Copyright:
© 2018 CEUR-WS. All Rights Reserved.

Recently viewed

Publications

  1. Do guided internet-based interventions result in clinically relevant changes for patients with depression?
  2. Understanding and Supporting Management Decision-Making
  3. Entity linking in 40 languages using MAG
  4. Unravelling insect declines: Can space replace time?
  5. Turning Good Intentions Into Actions by Using the Health Action Process Approach to Predict Adherence to Internet-Based Depression Prevention
  6. The generative drawing principle in multimedia learning
  7. Exploring transition research as transformative science
  8. A Lean Convolutional Neural Network for Vehicle Classification
  9. Recognition and approach responses toward threatening objects
  10. On the distinctiveness of tags in collaborative tagging systems
  11. An analytical approach to evaluating nonmonotonic functions of fuzzy numbers
  12. Space Systems Cross-Compatibility
  13. Understanding the properties of isospectral points and pairs in graphs
  14. NNARX networks on didactic level system identification
  15. Combining a PI Controller with an Adaptive Feedforward Control in PMSM
  16. Improvements in Flexibility depend on Stretching Duration
  17. Highly Efficient IPT Transmitter Circuit Based on a Novel Enhanced Class B Amplifier for Consumer Applications
  18. Retest effects in matrix test performance
  19. Multidimensionality of tree communities structure host-parasitoid networks and their phylogenetic composition
  20. Modernizing persistence–bioaccumulation–toxicity (PBT) assessment with high throughput animal-free methods
  21. Machine Learning and Data Mining for Sports Analytics
  22. Double-fading support - A training approach to complex software systems
  23. Time Use Research and Time Use Data