TY - CHAP

T1 - 9th challenge on question answering over linked data (QALD-9)

AU - Usbeck, Ricardo

AU - Gusmita, Ria Hari

AU - Ngomo, Axel Cyrille Ngonga

AU - Saleem, Muhammad

N1 - Funding Information: Acknowledgements This work has been supported by the H2020 project HOBBIT (GA no. 688227) as well as by the BMVI projects LIMBO (project no. 19F2029C) and OPAL (project no. 19F20284) as well as by the German Federal Ministry of Education and Research (BMBF) within ’KMU-innovativ: Forschung für die zivile Sicherheit’ in particular ’Forschung für die zivile Sicherheit’ and the project SOLIDE (no. 13N14456). The authors gratefully acknowledge financial support from the German Federal Ministry of Education and Research within Eurostars, a joint programme of EUREKA and the European Community under the project E! 9367 DIESEL and E! 9725 QAMEL. Publisher Copyright: © 2018 CEUR-WS. All Rights Reserved.

PY - 2018/10

Y1 - 2018/10

N2 - The QALD-9 challenge focused on the successful and long running multilingual QA task. In particular, we wanted to create an updated dataset which is larger than ever before and has a higher quality due to the employment of QUANT's semi-automatic curation measure. [2]. The teams were required to provide web-services of their systems to participate in the challenge, which will in turn support comparable research in the future in contrast to former XML/JSON file submissions. This increased the entrance requirements for participating teams but ensures long term comparability of the system performance and a fair and open challenge. In the future, we will further simplify the participation process and offer leaderboards prior to the actual challenge to allow participants to see their performance beforehand. After feedback from the authors, we will likely add new key performance indicators for the capability of a system to know which questions it cannot answer and take confidence scores for answers into account.

AB - The QALD-9 challenge focused on the successful and long running multilingual QA task. In particular, we wanted to create an updated dataset which is larger than ever before and has a higher quality due to the employment of QUANT's semi-automatic curation measure. [2]. The teams were required to provide web-services of their systems to participate in the challenge, which will in turn support comparable research in the future in contrast to former XML/JSON file submissions. This increased the entrance requirements for participating teams but ensures long term comparability of the system performance and a fair and open challenge. In the future, we will further simplify the participation process and offer leaderboards prior to the actual challenge to allow participants to see their performance beforehand. After feedback from the authors, we will likely add new key performance indicators for the capability of a system to know which questions it cannot answer and take confidence scores for answers into account.

KW - Informatics

KW - Business informatics

UR - https://www.scopus.com/record/display.uri?eid=2-s2.0-85056902191&origin=inward

UR - http://www.scopus.com/inward/record.url?scp=85056902191&partnerID=8YFLogxK

M3 - Article in conference proceedings

VL - 2241

T3 - CEUR Workshop Proceedings

SP - 58

EP - 64

BT - SemDeep-4_NLIWOD-4 2018

A2 - Choi, Key-Sun

A2 - Anke, Luis Espinosa

A2 - Decklerck, Thierry

A2 - Gromann, Dagmar

A2 - Kim, Jin-Dong

A2 - Ngomo, Axel-Cyrille Ngonga

A2 - Saleem, Muhammad

A2 - Usbeck, Ricardo

PB - Sun Site Central Europe (RWTH Aachen University)

T2 - Joint 4th Workshop on Semantic Deep Learning: Natural Language Interfaces for the Web of Data and 9th Question Answering over Linked Data Challenge, SemDeep-4_NLIWOD-4 2018

Y2 - 8 October 2018 through 9 October 2018

ER -