8th challenge on question answering over linked data (QALD-8)

Research output: Contributions to collected editions/worksArticle in conference proceedingsResearchpeer-review

Standard

8th challenge on question answering over linked data (QALD-8). / Usbeck, Ricardo; Ngomo, Axel Cyrille Ngonga; Conrads, Felix et al.
SemDeep-4_NLIWOD-4 2018: Joint Proceedings of ISWC 2018 Workshops SemDeep-4 and NLIWOD-4. ed. / Key-Sun Choi; Luis Espinosa Anke; Thierry Decklerck; Dagmar Gromann; Jin-Dong Kim; Axel-Cyrille Ngonga Ngomo; Muhammad Saleem; Ricardo Usbeck. Vol. 2241 Sun Site Central Europe (RWTH Aachen University), 2018. p. 51-57 (CEUR Workshop Proceedings; Vol. 2241).

Research output: Contributions to collected editions/worksArticle in conference proceedingsResearchpeer-review

Harvard

Usbeck, R, Ngomo, ACN, Conrads, F, Röder, M & Napolitano, G 2018, 8th challenge on question answering over linked data (QALD-8). in K-S Choi, LE Anke, T Decklerck, D Gromann, J-D Kim, A-CN Ngomo, M Saleem & R Usbeck (eds), SemDeep-4_NLIWOD-4 2018: Joint Proceedings of ISWC 2018 Workshops SemDeep-4 and NLIWOD-4. vol. 2241, CEUR Workshop Proceedings, vol. 2241, Sun Site Central Europe (RWTH Aachen University), pp. 51-57, Joint 4th Workshop on Semantic Deep Learning: Natural Language Interfaces for the Web of Data and 9th Question Answering over Linked Data Challenge, SemDeep-4_NLIWOD-4 2018, Monterey, California, United States, 08.10.18. <https://nbn-resolving.org/urn:nbn:de:0074-2241-6>

APA

Usbeck, R., Ngomo, A. C. N., Conrads, F., Röder, M., & Napolitano, G. (2018). 8th challenge on question answering over linked data (QALD-8). In K.-S. Choi, L. E. Anke, T. Decklerck, D. Gromann, J.-D. Kim, A.-C. N. Ngomo, M. Saleem, & R. Usbeck (Eds.), SemDeep-4_NLIWOD-4 2018: Joint Proceedings of ISWC 2018 Workshops SemDeep-4 and NLIWOD-4 (Vol. 2241, pp. 51-57). (CEUR Workshop Proceedings; Vol. 2241). Sun Site Central Europe (RWTH Aachen University). https://nbn-resolving.org/urn:nbn:de:0074-2241-6

Vancouver

Usbeck R, Ngomo ACN, Conrads F, Röder M, Napolitano G. 8th challenge on question answering over linked data (QALD-8). In Choi KS, Anke LE, Decklerck T, Gromann D, Kim JD, Ngomo ACN, Saleem M, Usbeck R, editors, SemDeep-4_NLIWOD-4 2018: Joint Proceedings of ISWC 2018 Workshops SemDeep-4 and NLIWOD-4. Vol. 2241. Sun Site Central Europe (RWTH Aachen University). 2018. p. 51-57. (CEUR Workshop Proceedings).

Bibtex

@inbook{590f00f8967f45c1b12abc36b2dc9df6,
title = "8th challenge on question answering over linked data (QALD-8)",
abstract = "The QALD-8 challenge focused on the successful and long running multilingual QA task. For the first time, the participating teams were required to provide webservices of their systems to participate in the challenge, which will in turn support comparable research in the future. In this challenge, we also changed the underlying evaluation platform to account for the need for comparable experiments via webservices in contrast to former XML/JSON file submissions. This increased the entrance requirements for participating teams but ensures long term comparability of the system performance and a fair and open challenge. In the future, we will further simplify the participation process and offer leaderboards prior to the actual challenge to allow participants to see their performance beforehand. After feedback from the authors, we will likely add new key performance indicators for the capability of a system to know which questions it cannot answer and take confidence scores for answers into account. Moreover, we will remove most of the curve ball questions to reflect the original character of the QALD challenge, which provides a clean and linguistically challenging benchmark.",
keywords = "Informatics, Business informatics",
author = "Ricardo Usbeck and Ngomo, {Axel Cyrille Ngonga} and Felix Conrads and Michael R{\"o}der and Giulio Napolitano",
note = "Publisher Copyright: {\textcopyright} 2018 CEUR-WS. All Rights Reserved.; Joint 4th Workshop on Semantic Deep Learning: Natural Language Interfaces for the Web of Data and 9th Question Answering over Linked Data Challenge, SemDeep-4_NLIWOD-4 2018 : co-located with the 17th International Semantic Web Conference ; Conference date: 08-10-2018 Through 09-10-2018",
year = "2018",
month = oct,
language = "English",
volume = "2241",
series = "CEUR Workshop Proceedings",
publisher = "Sun Site Central Europe (RWTH Aachen University)",
pages = "51--57",
editor = "Key-Sun Choi and Anke, {Luis Espinosa} and Thierry Decklerck and Dagmar Gromann and Jin-Dong Kim and Ngomo, {Axel-Cyrille Ngonga} and Muhammad Saleem and Ricardo Usbeck",
booktitle = "SemDeep-4_NLIWOD-4 2018",
address = "Germany",
url = "https://ceur-ws.org/Vol-2241/, https://ceur-ws.org/Vol-2241/JointPreface.pdf",

}

RIS

TY - CHAP

T1 - 8th challenge on question answering over linked data (QALD-8)

AU - Usbeck, Ricardo

AU - Ngomo, Axel Cyrille Ngonga

AU - Conrads, Felix

AU - Röder, Michael

AU - Napolitano, Giulio

N1 - Publisher Copyright: © 2018 CEUR-WS. All Rights Reserved.

PY - 2018/10

Y1 - 2018/10

N2 - The QALD-8 challenge focused on the successful and long running multilingual QA task. For the first time, the participating teams were required to provide webservices of their systems to participate in the challenge, which will in turn support comparable research in the future. In this challenge, we also changed the underlying evaluation platform to account for the need for comparable experiments via webservices in contrast to former XML/JSON file submissions. This increased the entrance requirements for participating teams but ensures long term comparability of the system performance and a fair and open challenge. In the future, we will further simplify the participation process and offer leaderboards prior to the actual challenge to allow participants to see their performance beforehand. After feedback from the authors, we will likely add new key performance indicators for the capability of a system to know which questions it cannot answer and take confidence scores for answers into account. Moreover, we will remove most of the curve ball questions to reflect the original character of the QALD challenge, which provides a clean and linguistically challenging benchmark.

AB - The QALD-8 challenge focused on the successful and long running multilingual QA task. For the first time, the participating teams were required to provide webservices of their systems to participate in the challenge, which will in turn support comparable research in the future. In this challenge, we also changed the underlying evaluation platform to account for the need for comparable experiments via webservices in contrast to former XML/JSON file submissions. This increased the entrance requirements for participating teams but ensures long term comparability of the system performance and a fair and open challenge. In the future, we will further simplify the participation process and offer leaderboards prior to the actual challenge to allow participants to see their performance beforehand. After feedback from the authors, we will likely add new key performance indicators for the capability of a system to know which questions it cannot answer and take confidence scores for answers into account. Moreover, we will remove most of the curve ball questions to reflect the original character of the QALD challenge, which provides a clean and linguistically challenging benchmark.

KW - Informatics

KW - Business informatics

UR - http://www.scopus.com/inward/record.url?scp=85056810696&partnerID=8YFLogxK

M3 - Article in conference proceedings

AN - SCOPUS:85056810696

VL - 2241

T3 - CEUR Workshop Proceedings

SP - 51

EP - 57

BT - SemDeep-4_NLIWOD-4 2018

A2 - Choi, Key-Sun

A2 - Anke, Luis Espinosa

A2 - Decklerck, Thierry

A2 - Gromann, Dagmar

A2 - Kim, Jin-Dong

A2 - Ngomo, Axel-Cyrille Ngonga

A2 - Saleem, Muhammad

A2 - Usbeck, Ricardo

PB - Sun Site Central Europe (RWTH Aachen University)

T2 - Joint 4th Workshop on Semantic Deep Learning: Natural Language Interfaces for the Web of Data and 9th Question Answering over Linked Data Challenge, SemDeep-4_NLIWOD-4 2018

Y2 - 8 October 2018 through 9 October 2018

ER -