GERBIL - General entity annotator benchmarking framework

Publikation: Beiträge in SammelwerkenAufsätze in KonferenzbändenForschungbegutachtet

Authors

  • Michael Röder
  • Axel Cyrille Ngonga Ngomo
  • Ciro Baron
  • Andreas Both
  • Martin Brümmer
  • Diego Ceccarelli
  • Marco Cornolti
  • Didier Cherix
  • Bernd Eickmann
  • Paolo Ferragina
  • Christiane Lemke
  • Andrea Moro
  • Roberto Navigli
  • Francesco Piccinno
  • Giuseppe Rizzo
  • Harald Sack
  • René Speck
  • Raphaël Troncy
  • Jörg Waitelonis
  • Lars Wesemann

The need to bridge between the unstructured data on the Document Web and the structured data on the Web of Data has led to the development of a considerable number of annotation tools. However, these tools are currently still hard to compare since the published evaluation results are calculated on diverse datasets and evaluated based on different measures. We present GERBIL, an evaluation framework for semantic entity annotation. The rationale behind our framework is to provide developers, end users and researchers with easy-To-use interfaces that allow for the agile, fine-grained and uniform evaluation of annotation tools on multiple datasets. By these means, we aim to ensure that both tool developers and end users can derive meaningful insights pertaining to the extension, integration and use of annotation applications. In particular, GERBIL provides comparable results to tool developers so as to allow them to easily discover the strengths and weaknesses of their implementations with respect to the state of the art. With the permanent experiment URIs provided by our framework, we ensure the reproducibility and archiving of evaluation results. Moreover, the framework generates data in machineprocessable format, allowing for the efficient querying and post-processing of evaluation results. Finally, the tool diagnostics provided by GERBIL allows deriving insights pertaining to the areas in which tools should be further refined, thus allowing developers to create an informed agenda for extensions and end users to detect the right tools for their purposes. GERBIL aims to become a focal point for the state of the art, driving the research agenda of the community by presenting comparable objective evaluation results.

OriginalspracheEnglisch
TitelWWW 2015 - Proceedings of the 24th International Conference on World Wide Web
HerausgeberAldo Gangemi, Stefano Leonardi, Alessandro Panconesi
Anzahl der Seiten11
VerlagAssociation for Computing Machinery, Inc
Erscheinungsdatum18.05.2015
Seiten1133-1143
ISBN (Print)978-1-4503-3469-3
DOIs
PublikationsstatusErschienen - 18.05.2015
Extern publiziertJa
Veranstaltung24th International Conference on World Wide Web, WWW 2015 - Florence, Italien
Dauer: 18.05.201522.05.2015
https://dl.acm.org/doi/proceedings/10.1145/2740908
https://dblp.org/db/conf/www/www2015.html

Links

DOI

Zuletzt angesehen

Publikationen

  1. Turning Good Intentions Into Actions by Using the Health Action Process Approach to Predict Adherence to Internet-Based Depression Prevention
  2. Pluralism and diversity: Trends in the use and application of ordination methods 1990-2007
  3. Deterministic Pod Repositioning in Robotic Mobile Fulfillment Systems
  4. Measurement and calculation of the viscosity of metals - A review of the current status and developing trends
  5. Machine Learning and Data Mining for Sports Analytics
  6. A latent state-trait analysis of current achievement motivation across different tasks of cognitive ability
  7. Efficacy of a web-based intervention with and without guidance for employees with risky drinking
  8. Durban as a Case in Point?
  9. Modeling of temperature- and strain-driven intermetallic compound evolution in an Al-Mg system via a multiphase-field approach with application to refill friction stir spot welding
  10. Home range size and resource use of breeding and non-breeding white storks along a land use gradient
  11. Quality System Development at the University of Graz
  12. Reporting and Analysing the Environmental Impact of Language Models on the Example of Commonsense Question Answering with External Knowledge
  13. Entity linking in 40 languages using MAG
  14. Nest site selection and the effects of land use in a multi-scale approach on the distribution of a passerine in an island arid environment
  15. Do consumers prefer pasture-raised dual-purpose cattle when considering meat products? A hypothetical discrete choice experiment for the case of minced beef
  16. Discourse, practice, policy and organizing
  17. FaQuAD
  18. Recurring patterns and blueprints of industrial symbioses as structural units for an it tool
  19. Determinants and Outcomes of Dual Distribution:
  20. Object-Oriented Construction Handbook
  21. Optimization of a gaseous multitube detector for soft X-ray detection
  22. A Multilab Replication of the Ego Depletion Effect
  23. A community of shared values? Dimensions and dynamics of cultural integration in the European Union
  24. Investigating the Promotional Effect of Green Signals in Sponsored Search Advertising Using Bayesian Parameter Estimation
  25. German Utilities and distributed PV
  26. Portrait of a Thinker
  27. Multi-view hidden markov perceptrons
  28. Diffusion of the Balanced Scorecard