GERBIL - General entity annotator benchmarking framework

Publikation: Beiträge in SammelwerkenAufsätze in KonferenzbändenForschungbegutachtet

Standard

GERBIL - General entity annotator benchmarking framework. / Usbeck, Ricardo; Röder, Michael; Ngomo, Axel Cyrille Ngonga et al.
WWW 2015 - Proceedings of the 24th International Conference on World Wide Web. Hrsg. / Aldo Gangemi; Stefano Leonardi; Alessandro Panconesi. Association for Computing Machinery, Inc, 2015. S. 1133-1143 (WWW 2015 - Proceedings of the 24th International Conference on World Wide Web).

Publikation: Beiträge in SammelwerkenAufsätze in KonferenzbändenForschungbegutachtet

Harvard

Usbeck, R, Röder, M, Ngomo, ACN, Baron, C, Both, A, Brümmer, M, Ceccarelli, D, Cornolti, M, Cherix, D, Eickmann, B, Ferragina, P, Lemke, C, Moro, A, Navigli, R, Piccinno, F, Rizzo, G, Sack, H, Speck, R, Troncy, R, Waitelonis, J & Wesemann, L 2015, GERBIL - General entity annotator benchmarking framework. in A Gangemi, S Leonardi & A Panconesi (Hrsg.), WWW 2015 - Proceedings of the 24th International Conference on World Wide Web. WWW 2015 - Proceedings of the 24th International Conference on World Wide Web, Association for Computing Machinery, Inc, S. 1133-1143, 24th International Conference on World Wide Web, WWW 2015, Florence, Italien, 18.05.15. https://doi.org/10.1145/2736277.2741626

APA

Usbeck, R., Röder, M., Ngomo, A. C. N., Baron, C., Both, A., Brümmer, M., Ceccarelli, D., Cornolti, M., Cherix, D., Eickmann, B., Ferragina, P., Lemke, C., Moro, A., Navigli, R., Piccinno, F., Rizzo, G., Sack, H., Speck, R., Troncy, R., ... Wesemann, L. (2015). GERBIL - General entity annotator benchmarking framework. In A. Gangemi, S. Leonardi, & A. Panconesi (Hrsg.), WWW 2015 - Proceedings of the 24th International Conference on World Wide Web (S. 1133-1143). (WWW 2015 - Proceedings of the 24th International Conference on World Wide Web). Association for Computing Machinery, Inc. https://doi.org/10.1145/2736277.2741626

Vancouver

Usbeck R, Röder M, Ngomo ACN, Baron C, Both A, Brümmer M et al. GERBIL - General entity annotator benchmarking framework. in Gangemi A, Leonardi S, Panconesi A, Hrsg., WWW 2015 - Proceedings of the 24th International Conference on World Wide Web. Association for Computing Machinery, Inc. 2015. S. 1133-1143. (WWW 2015 - Proceedings of the 24th International Conference on World Wide Web). doi: 10.1145/2736277.2741626

Bibtex

@inbook{789a0b175a6241adaacb83ae50c7b1b8,
title = "GERBIL - General entity annotator benchmarking framework",
abstract = "The need to bridge between the unstructured data on the Document Web and the structured data on the Web of Data has led to the development of a considerable number of annotation tools. However, these tools are currently still hard to compare since the published evaluation results are calculated on diverse datasets and evaluated based on different measures. We present GERBIL, an evaluation framework for semantic entity annotation. The rationale behind our framework is to provide developers, end users and researchers with easy-To-use interfaces that allow for the agile, fine-grained and uniform evaluation of annotation tools on multiple datasets. By these means, we aim to ensure that both tool developers and end users can derive meaningful insights pertaining to the extension, integration and use of annotation applications. In particular, GERBIL provides comparable results to tool developers so as to allow them to easily discover the strengths and weaknesses of their implementations with respect to the state of the art. With the permanent experiment URIs provided by our framework, we ensure the reproducibility and archiving of evaluation results. Moreover, the framework generates data in machineprocessable format, allowing for the efficient querying and post-processing of evaluation results. Finally, the tool diagnostics provided by GERBIL allows deriving insights pertaining to the areas in which tools should be further refined, thus allowing developers to create an informed agenda for extensions and end users to detect the right tools for their purposes. GERBIL aims to become a focal point for the state of the art, driving the research agenda of the community by presenting comparable objective evaluation results.",
keywords = "Archivability, Benchmarking Framework, Reusability, Semantic Entity Annotation System, Informatics, Business informatics",
author = "Ricardo Usbeck and Michael R{\"o}der and Ngomo, {Axel Cyrille Ngonga} and Ciro Baron and Andreas Both and Martin Br{\"u}mmer and Diego Ceccarelli and Marco Cornolti and Didier Cherix and Bernd Eickmann and Paolo Ferragina and Christiane Lemke and Andrea Moro and Roberto Navigli and Francesco Piccinno and Giuseppe Rizzo and Harald Sack and Ren{\'e} Speck and Rapha{\"e}l Troncy and J{\"o}rg Waitelonis and Lars Wesemann",
year = "2015",
month = may,
day = "18",
doi = "10.1145/2736277.2741626",
language = "English",
isbn = "978-1-4503-3469-3",
series = "WWW 2015 - Proceedings of the 24th International Conference on World Wide Web",
publisher = "Association for Computing Machinery, Inc",
pages = "1133--1143",
editor = "Aldo Gangemi and Stefano Leonardi and Alessandro Panconesi",
booktitle = "WWW 2015 - Proceedings of the 24th International Conference on World Wide Web",
address = "United States",
note = "24th International Conference on World Wide Web, WWW 2015, WWW 2015 ; Conference date: 18-05-2015 Through 22-05-2015",
url = "https://dl.acm.org/doi/proceedings/10.1145/2740908, https://dblp.org/db/conf/www/www2015.html",

}

RIS

TY - CHAP

T1 - GERBIL - General entity annotator benchmarking framework

AU - Usbeck, Ricardo

AU - Röder, Michael

AU - Ngomo, Axel Cyrille Ngonga

AU - Baron, Ciro

AU - Both, Andreas

AU - Brümmer, Martin

AU - Ceccarelli, Diego

AU - Cornolti, Marco

AU - Cherix, Didier

AU - Eickmann, Bernd

AU - Ferragina, Paolo

AU - Lemke, Christiane

AU - Moro, Andrea

AU - Navigli, Roberto

AU - Piccinno, Francesco

AU - Rizzo, Giuseppe

AU - Sack, Harald

AU - Speck, René

AU - Troncy, Raphaël

AU - Waitelonis, Jörg

AU - Wesemann, Lars

PY - 2015/5/18

Y1 - 2015/5/18

N2 - The need to bridge between the unstructured data on the Document Web and the structured data on the Web of Data has led to the development of a considerable number of annotation tools. However, these tools are currently still hard to compare since the published evaluation results are calculated on diverse datasets and evaluated based on different measures. We present GERBIL, an evaluation framework for semantic entity annotation. The rationale behind our framework is to provide developers, end users and researchers with easy-To-use interfaces that allow for the agile, fine-grained and uniform evaluation of annotation tools on multiple datasets. By these means, we aim to ensure that both tool developers and end users can derive meaningful insights pertaining to the extension, integration and use of annotation applications. In particular, GERBIL provides comparable results to tool developers so as to allow them to easily discover the strengths and weaknesses of their implementations with respect to the state of the art. With the permanent experiment URIs provided by our framework, we ensure the reproducibility and archiving of evaluation results. Moreover, the framework generates data in machineprocessable format, allowing for the efficient querying and post-processing of evaluation results. Finally, the tool diagnostics provided by GERBIL allows deriving insights pertaining to the areas in which tools should be further refined, thus allowing developers to create an informed agenda for extensions and end users to detect the right tools for their purposes. GERBIL aims to become a focal point for the state of the art, driving the research agenda of the community by presenting comparable objective evaluation results.

AB - The need to bridge between the unstructured data on the Document Web and the structured data on the Web of Data has led to the development of a considerable number of annotation tools. However, these tools are currently still hard to compare since the published evaluation results are calculated on diverse datasets and evaluated based on different measures. We present GERBIL, an evaluation framework for semantic entity annotation. The rationale behind our framework is to provide developers, end users and researchers with easy-To-use interfaces that allow for the agile, fine-grained and uniform evaluation of annotation tools on multiple datasets. By these means, we aim to ensure that both tool developers and end users can derive meaningful insights pertaining to the extension, integration and use of annotation applications. In particular, GERBIL provides comparable results to tool developers so as to allow them to easily discover the strengths and weaknesses of their implementations with respect to the state of the art. With the permanent experiment URIs provided by our framework, we ensure the reproducibility and archiving of evaluation results. Moreover, the framework generates data in machineprocessable format, allowing for the efficient querying and post-processing of evaluation results. Finally, the tool diagnostics provided by GERBIL allows deriving insights pertaining to the areas in which tools should be further refined, thus allowing developers to create an informed agenda for extensions and end users to detect the right tools for their purposes. GERBIL aims to become a focal point for the state of the art, driving the research agenda of the community by presenting comparable objective evaluation results.

KW - Archivability

KW - Benchmarking Framework

KW - Reusability

KW - Semantic Entity Annotation System

KW - Informatics

KW - Business informatics

UR - http://www.scopus.com/inward/record.url?scp=85018193235&partnerID=8YFLogxK

UR - https://www.mendeley.com/catalogue/d391ff27-3f0a-38ee-8fef-67b407a1719a/

U2 - 10.1145/2736277.2741626

DO - 10.1145/2736277.2741626

M3 - Article in conference proceedings

AN - SCOPUS:85018193235

SN - 978-1-4503-3469-3

T3 - WWW 2015 - Proceedings of the 24th International Conference on World Wide Web

SP - 1133

EP - 1143

BT - WWW 2015 - Proceedings of the 24th International Conference on World Wide Web

A2 - Gangemi, Aldo

A2 - Leonardi, Stefano

A2 - Panconesi, Alessandro

PB - Association for Computing Machinery, Inc

T2 - 24th International Conference on World Wide Web, WWW 2015

Y2 - 18 May 2015 through 22 May 2015

ER -

Links

DOI